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ABSTRACT 

Effective  and  efficient  decision-making  in  the  Australian  Army  is  integral  to  successful 
performance.  Therefore,  understanding  the  decision-making  process  is  a  research  priority.  In 
order  to  conduct  such  research,  a  summation  of  previous  knowledge  addressing  the  decision¬ 
making  process  and  the  climate  within  which  it  is  embedded,  is  important.  This  report  provides  a 
theoretical  platform  on  which  to  ground  future  research.  It  focuses  on  topics  such  as:  decision 
theories,  previous  experience  and  expert  functioning,  levels  of  expertise,  teamwork  and 
naturalistic  decision-making  and  decision  error.  The  interrelations  between  these  concepts  are 
discussed,  and  recommendations  for  future  research  are  given.  Future  research  would  be 
expected  to  inform  the  development  of  advanced  decision  support  tools,  and  superior  training 
techniques. 
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Expert  Decision-Making  in  Naturalistic 
Environments:  A  Summary  of  Research 


Executive  Summary 

The  importance  of  decision-making  in  the  military  has  provoked  much  research  interest. 
As  a  result  a  large  amount  of  relevant  research  directly  linked  with  command  decision¬ 
making  exists.  It  is  the  aim  of  this  report  to  summarise  and  comment  on  this  body  of 
literature,  specifically  in  regard  to  future  research  with  the  Australian  Army.  The  report 
has  been  divided  into  five  main  sections.  These  are:  decision  theories,  previous  experience 
and  expert  functioning,  levels  of  expertise,  teamwork  and  naturalistic  decision-making, 
and  decision  error. 

Firstly,  it  is  important  to  understand  the  ever-growing  field  of  decision-making  research. 
Decision  research,  as  a  field  of  Psychology,  dates  back  to  the  middle  of  the  20th  century. 
Classical  decision  theory  (Edwards,  1954)  and  the  rational  choice  model  (Janis  &  Mann, 
1977)  are  examples  of  widely  cited  research  that  has  focused  on  decision-making  as  an 
analytical-cognitive  process.  A  new  school  of  thought,  called  naturalistic  decision-making 
(NDM)  has  emerged,  which  is  thought  to  more  adequately  explain  decision-making  in 
emergency  situations.  Within  the  field  of  NDM,  the  attention  of  researchers  has  been 
focused  on  how  experts  make  decisions  in  real  life  emergency  situations. 

Secondly,  Commanders  in  the  military  can  all  claim  to  have  a  certain  amount  of 
experience  in  their  field.  Therefore,  understanding  the  concept  of  expertise  is  also 
important.  Research  on  novice-expert  differences  has  revealed  that  there  are  likely  to  be 
certain  cognitive  and  perceptual  skills  that  experts  have  developed  in  order  to  improve 
their  decision-making.  One  method  of  generating  a  greater  understanding  of  expert/ 
novice  differences  is  by  drawing  on  literature  from  the  field  of  cognitive  skill 
development.  Expertise  in  this  sense  can  be  considered  to  be  similar  to  any  other  cognitive 
skill.  It  is  acquired  through  quality  practice.  Therefore,  researchers  must  understand  the 
continuum  with  which  they  are  working,  when  talking  about  "novices"  in  comparison 
with  "experts". 

In  addition,  the  environment  within  which  a  Commander  is  immersed  is  important.  The 
team  surrounding  the  Commander  must  be  taken  into  account.  In  order  to  do  this  it  is 
important  to  understand  how  teams  function.  Social  psychologists  have  identified  both 
social  and  structural  characteristics  that  are  commonly  observed  in  groups  of  people 
working  together.  These  include:  cohesiveness,  socialisation,  team  norms,  roles  and 
communication  networks.  Recent  team  research  has  focused  on  the  importance  of  shared 
mental  models  (SMM)  in  effective  teams.  SMM  guide  how  team  members  interact,  and 
also  facilitate  decision-making  in  teams.  Also,  recently  it  has  been  suggested  that  for 
research  and  modelling  purposes  a  "team  mind"  can  be  established  as  analogous  to  the 
mind  of  one  person.  This  offers  a  range  of  implications  for  future  research  and  design. 


Finally,  it  is  important  to  be  able  to  discriminate  successful  and  unsuccessful  decisions  in 
any  study  of  decision-making.  It  would  be  expected  that  the  more  experience  a  person 
had,  the  more  successful  they  would  be  at  decision-making.  However  this  has  been  found 
to  be  incorrect.  It  seems  that  decision  error  can  be  attributed  to  any  of:  individual, 
organisational,  or  social  factors.  Developing  an  overall  understanding  of  how  decision 
error  may  occur  in  team  environments,  such  as  a  headquarters  (HQ)  team,  should  aid  in 
modelling  such  decision-making  situations. 

In  summary  the  above  factors  have  been  identified  as  important  areas  for  researchers  to 
consider  when  analysing  decision-making  in  military  HQ.  Interrelations  between  the 
aforementioned  concepts  are  discussed,  and  recommendations  for  future  research  are 
given. 
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1.  Setting  the  Scene 

On  January  22nd  1991,  during  the  Gulf  War,  an  eight-man  SAS1  team  known  as  Bravo-Two-Zero 
were  sent  on  a  mission  behind  enemy  lines.  Their  mission  was  to  remain  concealed  near  the  main 
supply  route  in  Western  Iraq  for  Id  days.  During  this  time  they  would  be  expected  to  sever  Iraqi 
fibre-optic  cables,  and  report  on  the  movement  of  scud  missile  launchers.  However,  due  to  resource 
limitations,  they  were  given  suboptimal  equipment.  They  also  received  vague  intelligence  reports. 
This  was  apparent  when  they  arrived  at  their  drop  off  point  to  find  it  only  about  200  meters  from  an 
Iraqi  anti-aircraft  stronghold.  This  had  not  been  reported  by  Intelligence.  Because  this  information 
was  crucial  to  the  success  of  the  mission,  the  group  attempted  to  contact  their  base  via  radio  and 
inform  them  of  the  new  situation.  It  was  at  this  time  that  they  realised  that  they  had  no  contact  with 
base.  It  was  later  discovered  they  had  been  given  the  wrong  radio  frequencies.  The  group  then  made 
the  decision  to  sit  out  until  a  liaison  arrived  in  24  hours.  Unfortunately,  that  was  too  long  a  wait. 
They  were  spotted  by  an  Iraqi  goat  herder,  and  from  then  a  malady  of  errors  began  its  course.  The 
team  were  separated.  They  were  confronted  with  intermittent  enemy  contact,  and  were  completely 
unsupported  by  the  larger  organisation.  Over  the  next  three  days,  three  of  the  eight  died,  and  four 
were  captured.  Only  one  man  made  his  way  back  across  the  Syrian  border.  This  planning  error 
resulted  in  one  of  the  most  costly  patrols  in  SAS  history. 

(David,  1997) 

An  anecdote  such  as  this  indicates  the  importance  of  planning  and  decision-making  in 
such  circumstances.  To  fail  on  such  an  occasion  is  to  perish.  Therefore  the  stakes  are  high, 
and  the  priority  for  researchers  to  understand  the  decision-making  process  in  such 
circumstances  is  paramount.  This  report  aims  to  investigate  current  understandings  of 
expert  decision-making  in  naturalistic  environments,  such  as  the  one  described  in  the 
preceding  anecdote.  This  report  sets  out  the  relevant  background  information  required  to 
begin  to  understand  the  field  of  naturalistic  decision-making  (NDM).  It  also  describes  how 
expertise  is  understood  within  this  field,  and  how  an  expert  will  commonly  function  as  a 
part  of  a  team.  Finally,  this  report  identifies  important  areas  for  future  research  within  the 
NDM  domain. 


2.  Overview  of  Issues 


Due  to  the  large  amount  of  relevant  research  directly  linked  with  command  decision¬ 
making,  the  author  feels  it  is  important  to  provide  an  overview  of  the  relevant  points  at 
this  stage.  This  brief  summary  establishes  the  focal  areas  to  be  covered  more  substantially 
in  the  following  review. 


1  Note  this  is  the  UK  SAS,  not  the  Australian  SAS. 
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Decision  research  as  a  field  of  Psychology  dates  back  to  the  middle  of  the  20th  century. 
Classical  decision  theory  (Edwards,  1954)  and  the  rational  choice  model  (Janis  &  Mann, 
1977)  are  examples  of  widely  cited  research  that  has  focused  on  decision-making  as  an 
analytical-cognitive  process.  Although  classical  decision  theory  has  proved  useful  in 
describing  decision-making  in  a  number  of  situations,  it  seems  that  when:  timeframes  are 
short,  there  is  substantial  uncertainty  and  the  stakes  are  high,  an  alternative  theory  is 
needed.  In  response  to  this,  a  new  school  of  thought  has  emerged  called  naturalistic 
decision-making  (Klein,  Calderwood,  &  Clinton-Cirocco,  1986).  The  NDM  approach  aims 
to  investigate  decision-making  in  naturalistic  environments  where  uncertainty,  time 
pressure  and  high  stakes  are  key  variables.  This  theory  has  been  applied  to  domains  such 
as  fire  fighting,  defence  operations,  and  emergency  medicine.  Within  the  field  of  NDM,  the 
attention  of  researchers  has  been  focused  on  how  experts  in  real-life  situations  make 
decisions. 

The  study  of  expertise  is  not  limited  to  NDM  theory.  There  is  evidence  indicating  that, 
even  in  ancient  times,  Socrates  and  Descartes  took  it  for  granted  that  "understanding  a 
domain  means  holding  a  theory  of  that  domain"  (Dreyfus,  1997).  A  particular  challenge  in 
the  study  of  "expertise"  is  to  provide  an  adequate  definitionfor  this  term.  To  date,  it  seems 
that  an  accepted  definition  of  an  expert  in  the  literature  is  held  as  being  a  person  with  10 
or  more  years  experience  in  their  chosen  domain  (Klein,  1997). 

Understanding  how  an  expert  functions  is  thought  to  be  the  key  to  duplicating  expert 
performance  via  training.  Obviously  this  is  still  an  important  goal  in  the  field  of  NDM.  The 
understanding  of  cognitive  and  perceptual  skills  that  experts  may  have  developed  to 
improve  their  decision-making  has  been  improved  as  a  result  of  studies  comparing  experts 
with  novices  (Glaser,  1987). 

One  method  of  generating  a  greater  understanding  of  expert-novice  differences  is  by 
seeking  to  understand  cognitive  skill  development.  Expertise  in  this  sense  can  be 
considered  to  be  similar  to  any  other  cognitive  skill.  It  is  acquired  through  quality  practice. 
Therefore,  researchers  must  understand  the  continuum  they  are  working  with  when 
talking  about  "novices"  in  comparison  with  "experts".  Investigating  cognitive  skill 
development  also  provides  an  opportunity  to  understand  how  expertise  develops,  and 
such  an  understanding  may  aid  in  the  design  of  training  systems. 

Due  to  the  complexity  of  NDM  domains,  experts  will  often  work  in  teams  to  accomplish 
goals.  Therefore,  in  order  to  understand  how  a  particular  expert  functions  it  is  important 
to  consider  the  wider  team  context  in  which  they  are  immersed.  This  requires  a  review  of 
the  group  and  teamwork  literature  in  order  to  identify  how  teams  function.  A  number  of 
teamwork  theories  exist  and  these  are  outlined  in  the  report. 

In  any  study  of  decision-making,  it  is  important  to  be  able  to  discriminate  successful  and 
unsuccessful  decisions.  It  would  be  expected  that  the  more  experience  a  person  had,  the 
more  successful  they  would  be  at  decision-making.  "Expert  error"  seems  like  an 
oxymoron,  but  evidence  has  shown  that,  despite  extensive  domain  experience,  errors  can 
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still  occur  (NTSB,  1994).  Researchers  have  traditionally  attempted  to  understand  error  by 
searching  for  a  breakdown  in  expert  cognitive  processing.  However,  there  has  recently 
been  a  change  in  direction,  whereby  error  is  not  thought  of  as  caused  by  a  breakdown  in 
expert  processing,  but  by  a  poor  system  in  which  the  expert  is  immersed  (Reason,  1990; 
Woods,  Johannesen,  Cook,  &  Sarter,  1994).  Therefore,  the  relation  between  context  and 
expert  decision-making  has  been  identified  as  an  important  area  for  future  research 
(Lewandowsky  &  Kirsner,  2000;  Orasanu,  Martin  &  Davidson,  2001;  Schliemann  & 
Carraher,  1993).  Some  notable  work  that  has  already  been  done  on  the  effect  of  the  context 
on  decision-making  is  in  the  field  of  decision  biases  in  relation  to  decision  error  (Pious, 
1993).  It  is  also  suggested  that  there  may  be  social  and  organisational  considerations  that 
contribute  to  team  decision  error.  Examples  of  these  are  false  consensus  effect,  groupthink, 
group  polarisation,  and  group  escalation  of  commitment  (Jones  &  Roelofsma,  2000). 
Developing  an  overall  understanding  of  how  decision  error  may  occur  in  team 
environments,  such  as  a  headquarters  (HQ)  team,  should  aid  in  modelling  such  decision¬ 
making  situations. 

These  main  concepts  will  now  be  discussed  in  more  detail. 


3.  Decision  Theories 


Decision-making  is  an  important  cognitive  process  that  occurs  frequently  in  everyday 
human  functioning.  Decision-making  has  been  defined  by  Y ates  (2001)  as  the  process  that 
leads  to  the  commitment  to  an  action,  the  aim  of  which  is  to  produce  satisfying  outcomes. 
There  have  been  a  number  of  models  created  to  explain  this  decision-making  process. 
Preceding  the  development  of  the  field  of  NDM,  there  was  a  rich  tradition  of  research  and 
formulation  of  analytical  decision-making  models.  For  the  purpose  of  the  current 
summary,  prescriptive  and  descriptive  models  of  analytical  decision-making  are  first 
described.  Following  this  an  explanation  and  critique  of  models  within  the  field  of  NDM 
will  be  presented 

3.1  Prescriptive  (Analytical)  Decision-making 

Edwards  (1954)  introduced  the  classical  concept  of  prescriptive,  analytical  decision¬ 
making  to  the  field  of  Psychology.  This  is  a  prescriptive  decision-making  theory  that  can 
be  used  to  investigate  everyday  decision-making  processes.  The  theory  suggests  that 
people  should  collect  and  analyse  information,  eventually  selecting  an  optimal  solution 
from  a  range  of  alternatives.  This  should  be  done  by  evaluating  the  advantages  and 
disadvantages  of  each  possible  outcome  and  then  choosing  the  one  most  appropriate  to 
achieve  the  desired  outcome  goal.  This  decision  is  regarded  as  optimal  (McDaniel,  1993). 

This  form  of  decision-making  research  essentially  investigates  the  quality  of  the  decision 
made  by  comparing  it  with  the  outcome  predicted  by  a  normative  statistical  model.  In 
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doing  so,  this  theory  draws  upon  such  methods  as  expected  utility  theory  and  Bayesian 
analysis.  Possible  decision  outcomes  can  be  presented  statistically  as  probability  estimates 
for  expected  outcomes.  Relevant  research  has  relied  on  laboratory  experiments  to 
determine  how  the  optimal  decision  should  be  reached  instead  of  studying  how  people 
actually  make  decisions.  This  approach  has  also  examined  how  real-life  decision-making 
outcomes  differ  from  the  optimal  solution  (Brehmer,  Jungermann,  Lourens  &  Sevon,  1985). 

It  has  been  argued  that  this  theory  of  decision-making  has  weaknesses  when  applied  to 
real-life  decision-making  (Beach  &  Lipshitz,  1997).  In  particular,  it  has  been  suggested  that 
in  an  NDM  situation  this  model  of  decision-making  does  not  adequately  describe  the 
decision  process  (Klein,  1989).  Some  obvious  problems  within  the  NDM  environment  that 
cannot  be  catered  for  in  this  model  are  that  the  time  frames  in  real-life  situations  may  be 
short,  and  hence  evaluation  of  pros  and  cons  associated  with  a  variety  of  options  is  often 
not  possible.  In  fact,  recent  research  by  Klein,  Calderwood  and  Clinton-Cirrocco  (1997)  has 
found  that,  in  contrast  with  the  classical  decision-making  models,  on  average  fire-fighters 
only  focus  on  one  or  two  options  before  making  decisions.  Also,  in  NDM  environments 
quality  of  information  is  usually  ambiguous,  so  that  past  experience  and  the  ability  to 
interpret  information  is  important.  Thus  the  adequacy  of  classical  decision  theory  to 
explain  decision-making  in  an  NDM  environment  is  questionable. 

3.2  Descriptive  (Analytical)  Decision-making 

Following  the  failure  of  classical  decision  theory  to  describe  many  decision-making  events 
adequately,  a  number  of  descriptive  decision  theories  arose  (Pious,  1993).  One  of  these  is 
the  rational  choice  model  of  decision-making  (Janis  &  Mann,  1977).  This  model  provides 
an  analytical  approach  for  understanding  decision-making,  based  on  the  researchers' 
observations.  The  seven  process  criteria  specified  by  the  rational  choice  model  are: 

1.  A  wide  range  of  objectives  are  identified. 

2.  A  broad  spectrum  of  alternatives  are  produced. 

3.  The  advantages  and  disadvantages  of  each  alternative  are  assessed. 

4.  A  search  for  new  information  relevant  to  each  alternative  is  undertaken. 

5.  New  information  is  incorporated  into  the  analysis  of  the  alternatives. 

6.  The  advantages  and  disadvantages  are  revised. 

7.  Detailed  provisions  are  made  when  implementing  the  chosen  decision. 

Janis  and  Mann  (1977)  argued  that  this  model  would  be  the  only  way  that  individuals 
under  pressure  could  make  good  decisions.  One  pressure  that  was  discussed  by  Janis  and 
Mann  (1977)  was  time  pressure,  which  is  of  particular  interest  because  it  is  commonly  a 
characteristic  of  NDM  environments.  Associated  with  time  pressure,  Janis  and  Mann 
(1977)  discussed  how  errors  could  occur  due  to  people  being  in  a  "hypervigilant"  state2. 
Hypervigilance  was  associated  with  a  heightened  level  of  emotion,  and  it  was  suggested 
that  this  high  level  of  emotion  may  be  sustained  for  minutes  and  even  hours  after  the 


2  This  was  described  as  an  aroused  psychological  state  triggered  by  time  pressure. 
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decision  had  been  made.  Janis  and  Mann  (1977)  indicated  that  people  in  a  "hypervigilant 
state"  become  preoccupied  with  the  losses  that  may  occur  due  to  their  time  pressured 
decision-making.  They  also  suggested  that  decision-makers  will  be  indiscriminately  open 
to  all  information  available,  being  unable  to  discriminate  relevant  from  irrelevant.  This 
suggestion  does  not  take  into  account  the  experience  drawn  on  by  expert  decision-makers, 
and  information  searching  strategies  suggested  by  Klein  (1998).  However  it  does  highlight 
the  possible  association  between  time  pressure  and  increased  levels  of  emotion. 

To  date,  the  role  played  by  emotion  in  decision-making  has  not  been  given  adequate 
attention  in  the  NDM  literature.  Recent  research  in  the  field  of  neuropsychology  suggests 
that  emotions  may  play  a  profound  role  in  decision-making  (Lawrence  &  Nohira,  2002, 
cited  in  Masters  &  Mcshane,  2003).  It  has  been  suggested  that  the  brain's  emotional  centres 
actually  receive  signals  before  they  reach  the  cognitive  centres  of  the  brain.  The  emotional 
centres  will  process  input  more  quickly  and  tag  this  with  emotional  responses.  Following 
this,  the  stimuli  are  transported  to  the  cognitive  centre  where  they  will  be  processed  more 
logically.  Thus  emotion  seems  to  be  inextricably  linked  with  decision-making.  However, 
these  ideas  remain  largely  speculative  and  there  is  as  yet  little  direct  evidence  for  them. 

In  sum,  the  rational  choice  model  seems  to  describe  some  decision  events  and  Janis  and 
Mann  (1977)  have  highlighted  emotion  as  a  potential  influence  on  decision-making. 
Rational  choice  reasoning  currently  seems  to  provide  the  template  for  the  Military 
Appreciation  Process3  (MAP,  see  Australian  Army,  2004).  However  researchers  have 
suggested  that  much  human  decision-making  does  not  follow  the  rational  choice  model 
(eg.  Beach  &  Lipshitz,  1997).  In  particular,  the  finding  that  decision-makers  do  not  usually 
compare  a  wide  range  of  alternatives  compels  researchers  to  look  more  broadly  at  the 
decision-making  process.  For  example,  in  an  Army  situation  there  may  be  only  one 
solution  generated.  Also,  there  may  be  inadequate  time  to  search  for  new  information 
relevant  to  possible  alternatives,  and  to  incorporate  this  into  the  evaluation.  Instead  it  has 
been  suggested  that  the  past  experience  of  the  decision-maker  is  vital  in  this  context 
(Klein,  1998). 

Other  descriptive  analytical  theories  have  also  been  suggested  as  explanations  for 
decision-making.  Deviations  from  the  outcomes  predicted  by  traditional  analytic  decision 
models  have  illustrated  that  biases  are  present  in  human  decision-making.  From  this,  the 
decision  bias  and  heuristic  paradigm  emerged  (Pious,  1993).  Researchers  in  this  field 
attempt  to  describe  the  heuristics  that  human  decision-makers  tend  to  use  in  order  to 
reduce  cognitive  load.  A  summary  of  some  of  the  popular  decision  bias  theories  is 
provided  in  Table  1. 


3  The  Military  Appreciation  Process  is  a  series  of  steps  taught  to  Officers  to  apply  to  planning/ 
decision-making  scenarios. 
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Table  1.  Description  of  decision  theories  based  on  heuristics 


Decision  Theory 

Description 

Satisficing 
(Simon,  1956) 

To  choose  a  decision  option  that  satisfies  your  most 
important  needs.  Not  necessarily  the  optimal  path. 

Prospect  theory 
(Kahneman  &  Tversky,  1979) 

To  evaluate  alternatives  in  relation  to  a  reference 
point.  Within  this  theory  it  has  been  found  that,  in 
relation  to  that  point,  perceived  losses  are  weighted 
more  strongly  than  gains. 

The  certainty  effect 
(Allais,  1953) 

Where  alternatives  reduce  potential  risks  by  the  same 
amount,  the  alternative  that  reduces  risk  to  zero  is 
favoured. 

Pseudocertainty 
(Tversky  &  Kahneman,  1981) 

When  an  alternative  appears  to  reduce  the  risk  to 
zero,  it  is  more  popular. 

Regret  theory 
(Bell,  1982;  Dunning  & 

Parpal,  1989;Loomes  & 
Sugden,  1982) 

This  rests  on  two  assumptions.  Firstly,  that  many 
people  experience  regret  and  rejoicing,  and  secondly, 
that  we  take  this  into  account  when  we  make  a 
decision.  Therefore,  similar  to  prospect  theory,  we  try 
to  minimise  losses  (regret).  This,  however,  does  not 
hold  for  all  decision  cases. 

Multi-attribute  choice 
(Einhorn  &  Hogarth,  1981) 

Where  the  alternatives  attributes  are  weighted  and 
one  attribute  can  be  traded  off  for  another  more 
important  one. 

Non-compensatory  strategies 
(Hogarth,  1987) 

Similar  to  multi-attribute  choice,  but  choices  are  made 
beforehand  on  the  value  of  the  attributes,  and  trade 
offs  do  not  occur. 

The  more  important 
dimension 
(Slovic,  1975) 

When  people  are  faced  with  an  equal  choice  they  will 
choose  the  alternative  that  rates  highest  on  the  "most 
important  dimension". 

Firstly,  the  structure  of  the  studies  used  to  reveal  these  biases  has  been  criticised  as  being 
biased  to  producing  certain  results  (see  Lopes,  1991).  Secondly,  the  theories  do  not 
account  for  uncertain  information  and  they  have  not  been  tested  in  NDM  environments. 
Klein  (1998)  suggests  that  several  studies  found  that  decision  biases  are  reduced  if  the 
study  includes  contextual  factors,  and  that  decision  biases  do  not  occur  in  experienced 
decision-makers  working  in  natural  settings  (Christensen-Szalanski  &  Beach,  1984;  Fraser, 
Smith  &  Smith,  1992;  Gigerenzer,  1987;  Shanteau,  1992;  Smith  &  Kida,  1991).  It  seems  that, 
particularly  in  situations  characterised  by  time  pressure,  uncertainty,  complexity  and  high 
stakes,  analytical  decision  models  fail  to  provide  adequate  answers  (Salas,  Bowers  & 
Cannon  Bowers,  1995). 
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Overall,  these  theories  have  offered  some  valuable  explanations  for  observable 
phenomena,  but  each  has  fallen  short  of  providing  a  theory  for  decision-making  applicable 
to  naturalistic  situations  (where  there  are  often  multiple  goals,  uncertainty,  and  time 
pressure;  Fishchoff,  1988).  Other  descriptive  analytical  theories  have  also  been  suggested 
as  explanations  for  decision-making.  These  theories  have  been  largely  based  on  biases 
found  in  decision-making  (Pious,  1993).  They  attempt  to  describe  the  heuristics  that 
human  decision-makers  tend  to  use  in  order  to  reduce  cognitive  load.  Overall,  these 
theories  have  offered  some  valuable  explanations  for  observable  phenomena,  but  each  has 
fallen  short  of  providing  a  theory  for  decision-making  applicable  to  naturalistic  situations 
(where  there  are  often  multiple  goals,  uncertainty,  and  time  pressure).  They  tend  to  rely  on 
analytical  principles,  do  not  account  for  uncertain  information  and  have  not  been  tested  in 
NDM  environments. 

Analytical  models  of  decision-making  are  based  on  a  set  of  assumptions  that  decision¬ 
making  is  mechanistic,  and  based  on  probability  (Noakes,  in  press).  However,  evidence 
suggests  that  humans  do  not  consistently  use  probability-based  assumptions  (Allias  & 
Hagan,  1979;  Beach,  1966;  DuCharme,  1970;  Edwards,  1968;  Kahneman  &  Tversky,  1973; 
Savage,  1954).  It  has  been  suggested  that  needing  to  evaluate  such  a  large  number  of 
alternatives  in  a  short  period  of  time  is  impractical  in  naturalistic  situations  (Kaempf  & 
Militello,  1991).  In  fact  usually  only  one  option  is  considered  (Klein  et.  al,  1989).  Also, 
because  of  the  dynamic  environment,  continual  evaluation  and  judgement  of  the 
environment  is  required.  This  is  in  contrast  to  a  singular  moment  of  choice  assumed  by 
these  models.  Work  by  Janis  and  Mann  (1977)  has  successfully  brought  to  researchers' 
attention  the  potential  role  of  emotion  in  decision-making,  and  this  needs  to  be  further 
explored  in  an  NDM  context.  Finally,  naturalistic  environments  usually  require  expert 
decision-makers,  whose  experience  facilitates  their  decision-making.  This  fact  is  not 
adequately  explained  by  analytical  decision  theories.  Thus,  in  naturalistic  circumstances, 
there  are  grounds  for  querying  the  adequacy  of  analytical  decision  models. 

Moreover,  Ebbensen  &  Konecni  (1980)  have  argued  that  the  external  validity  of  decision¬ 
making  research  that  relies  on  laboratory  experiments  (such  as  many  of  the  analytical 
decision-making  models)  is  minimal.  It  may  be  that  biases  found  in  the  laboratory  are 
simply  caused  by  participants  not  understanding  the  task  (Fischkoff,  1988).  Application  of 
these  theories  to  other  environments,  and  real-world  decision-makers,  still  needs  to  be 
achieved.  Therefore  these  theories  still  require  extension,  in  order  to  describe  decision¬ 
making  adequately  in  a  military  context. 

3.2.1  Summarising  Analytical  Decision-making 

In  summary,  analytical  decision-making  theories,  such  as  classical  decision-making  theory, 
are  based  on  evaluating  options  and  are  used  to  select  statistically  optimal  solutions. 
However,  research  has  shown  that  humans  do  not  always  behave  in  this  way.  As  a 
consequence,  a  number  of  descriptive  decision  theories  were  developed  in  an  attempt  to 
account  for  some  of  the  biases  observed  in  decision-making.  While  these  theories  provided 
descriptions  of  certain  observable  phenomena,  they  still  tend  to  rely  on  evaluation  of 
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options  in  the  decision-making  process.  It  has  been  suggested  that  in  naturalistic 
environments,  constraints  such  as  time  pressure,  high-stakes  and  uncertainty  create  an 
environment  where  the  evaluation  of  options  is  not  possible. 

3.3  Descriptive  Naturalistic  Decision-Making  Theories 

In  response  to  the  inadequacy  of  analytical  decision-making  theories  to  explain  decision¬ 
making  in  many  real-world  decision-making  environments,  researchers  shifted  their  focus 
to  explore  the  unique  aspects  of  these  environments  They  were  termed  naturalistic 
environments,  and  included  domains  such  as  military,  firefighting  and  other  emergency 
response  organisations.  These  kinds  of  environments  have  been  defined  by  Omodei  and 
Wearing  (1995)  as  incorporating  the  following  conditions: 

•  Dynamicity  -  This  refers  to  the  continual  complexity  of  the  environment.  NDM 
involves  a  series  of  decisions,  rather  than  a  single  moment  of  choice  as  described  in 
traditional  decision-making  theory  (Brehmer,  1990).  In  this  environment,  each  decision 
made  affects  each  consequent  decision. 

•  Uncertainty  -  The  quality  of  available  information  about  a  situation  is  often 
questionable. 

•  Distribution  of  tasks  -  This  is  required  because  of  the  complexity  of  the  situation. 
There  is  too  much  information  for  an  individual  decision-maker  to  cope  with,  so  the 
decision  task  is  distributed  among  several  contributors 

3.3.1  Naturalistic  Decision-Making  Definition 

The  search  for  a  more  appropriate  model  of  decision-making  under  these  conditions  has 
seen  the  development  of  NDM  theory  (Cannon-Bowers,  Salas  &  Pruitt,  1996).  Zsambok 
(1997)  defines  NDM  as: 

. .  .how  experienced  people,  working  as  individuals  or  groups  in  dynamic,  uncertain, 
and  often  fast  paced  environments,  identify  and  assess  their  situation,  make  decisions 
and  take  actions  whose  consequences  are  meaningful  to  them  and  to  the  larger 
organisation  in  which  they  operate  (p.  5). 

In  other  words,  NDM  research  investigates  how  people  use  experience  to  make  decisions 
in  naturalistic  environments  (eg.  under  time  pressure,  shifting  conditions,  unclear  goals, 
degraded  information  and  within  team  interactions).  A  common  theme  in  NDM  research 
is  the  role  of  expertise  in  decision-making. 

Obviously,  the  environment  described  presents  a  specific  set  of  conditions  to  be 
considered  when  attempting  to  make  decisions.  It  is  also  understood  that  this  kind  of 
environment  is  not  unique  to  the  Army;  similar  environmental  demands  will  be  required 
of  fire-fighters,  workers  in  emergency  medicine,  and  air  traffic  controllers  among  others. 


8 


DSTO-GD-0429 


As  foreshadowed  above,  decision-making  within  real-world  environments  has  been 
described  as  naturalistic  decision-making  (Cannon-Bowers,  Salas  &  Pruitt,  1996).  In  other 
words,  NDM  research  investigates  how  people  use  experience  to  make  decisions  in 
naturalistic  environments  (eg.  under  time  pressure,  shifting  conditions,  with  unclear  goals, 
degraded  information  and  within  team  interactions).  Consistent  with  this  definition, 
research  has  aimed  to  identify  appropriate  models  of  decision-making  to  be  applied  to  this 
context.  Ideas  that  have  been  put  forward  include:  image  theory  (Beach  &  Mitchell,  1987), 
the  RPDM  model  (Klein,  et.  al,  1989),  and  mental  models  and  schema  theory  of  decision¬ 
making  (Lipshitz  &  Shaul,  1997).  These  will  now  be  critically  considered. 

3.3.2  Image  Theory 

Image  theory  is  an  example  of  a  descriptive  model  of  decision-making  that  has  been 
applied  successfully  in  some  real-world  settings  (auditing  decisions;  Beach  &  Fredrickson, 
1987,  cited  in  Beach  &  Mitchell,  1987).  In  contrast  to  traditional  decision  models  that  tend 
to  regard  information  as  the  only  determinant  for  decision-making,  image  theory  posits 
that  other  factors  should  be  considered.  It  focuses  on  an  individual's  values  and  goals  in 
combination  with  current  information.  It  also  posits  that  previous  experience  is  important 
for  decision-making.  This  idea  is  supported  by  other  research  (eg.  Klein,  1998). 

Image  theory  holds  that  the  decision-maker  should  possess  three  "decision-related 
images",  and  these  should  act  to  constrain  the  decisions  that  can  be  made.  These  images 
are  described  as: 

•  Value  image  -  This  image  contains  the  decision-maker's  principles;  his/her 
assumptions  about  how  things  should  be.  When  participating  in  a  group  the  decision¬ 
maker  can  participate  effectively  by  using  the  organisation's  values  when  making 
decisions  (Beach,  Smith,  Lundell  &  Mitchell,  1988). 

•  Trajectory  image  -  This  image  contains  the  goals  of  the  decision-maker  or  the  larger 
organisation  within  which  s/he  functions. 

•  Strategic  image  -  This  image  contains  the  strategies  and  plans  that  have  been  adopted 
to  accomplish  the  goals.  Each  plan  is  abstract;  a  sequence  of  potential  activities  to  work 
towards  eventual  goal  attainment.  Potential  plans  tend  to  come  from  past  experience. 

Along  with  these  images  a  decision-maker  will  have  goals  that  will  represent  the  desired 
outcome.  Combination  of  the  goals  and  the  values  is  required  for  the  production  of  a  plan. 
During  the  decision-making  process  it  is  suggested  that  a  compatibility  test  occurs  in  the 
decision-maker's  mind  to  examine  the  goodness-of-fit  between  the  potential  goals  and 
plans,  and  the  values.  Decision-making  therefore  requires  accepting  or  rejecting  goals  and 
plans  according  to  suitability  and  then  profitability.  In  this  way  it  is  still  incorporating  an 
analytical  approach  to  decision-making. 


9 


DSTO-GD-0429 


Because  goals  and  plans  are  linked  in  memory,  it  is  often  the  case  that  once  a  goal  has  been 
identified,  an  immediate  plan  of  action  can  be  applied.  The  link  between  goal  and  action  is 
called  a  "policy",  and  the  process  is  referred  to  as  "framing  the  decision".  It  involves 
probing  memory  with  information  from  the  current  situation.  If  there  is  a  similar 
contextual  memory  to  the  current  situation,  the  situation  is  recognised,  and  an  appropriate 
course  of  action  already  stored  can  be  modified  and  executed.  A  major  part  of  the 
decision-making  process  is  monitoring  the  courses  of  action  that  have  been  decided  upon. 

It  has  been  suggested  by  Beach  et  al.  (1988)  that  this  theory  could  be  applied  to  decision¬ 
making  in  naturalistic  environments.  However,  the  theory  seems  quite  difficult  to  validate, 
and  at  this  time  has  not  been  applied  successfully  to  a  NDM  environment.  A  number  of 
studies  have  aimed  to  test  aspects  of  the  theory.  For  example,  laboratory  research  has 
supported  the  ideas  of  compatibility  and  profitability  tests  during  decision-making 
(Nicholas-Hoppe  &  Beach,  1987;  Payne,  1976).  Also,  Dunegan  (1995)  has  found  that  when 
perceptions  of  a  project's  current  and  target  images  were  more  similar  decision-makers 
tended  to  continue  with  that  project.  Finally,  the  model  has  been  applied  in  the  real  world 
to  situations  such  as  organisational  decision-making,  decision-making  for  married  couples 
and  also  auditing  decisions  (Mitchell,  Rediker  &  Beach,  1986;  Beach  &  Morrison,  1987; 
Beach  &  Fredrickson,  1987,  cited  in  Beach  &  Mitchell,  1987).  Thus  there  has  been  some 
support  for  this  theory,  but  further  research  is  required  in  order  to  validate  the  theory  and 
to  test  empirically  whether  it  adequately  describes  decision-making  in  naturalistic 
environments.  Application  to  other  environments,  and  real-world  decision-makers,  still 
needs  to  be  achieved.  Moreover  the  model  does  not  integrate  emotion  into  the  decision¬ 
making  explanation. 

3.3.3  The  Recognition  Primed  Decision-making  Model 

A  well-cited  model  of  decision-making,  actually  developed  from  observations  in 
naturalistic  environments,  is  the  Recognition  Primed  Decision-making  (RPD)  model  (Klein 
et  al.,  1986).  The  model  was  based  on  interviews  and  observations  of  fire  ground 
commanders  working  in  difficult  and  challenging  circumstances  (Klein  et  al.,  1986).  It  was 
anticipated  that,  under  complex  circumstances  involving  time  pressures,  the  Commanders 
would  make  limited  comparisons  between  possible  outcomes.  However,  the  early  research 
revealed  that  they  were  making  no  comparisons  at  all.  In  fact,  80%  of  decisions  made  in 
this  task  were  made  in  less  than  one  minute.  From  this,  Klein  et  al.  (1986)  identified  the 
following  issues: 

•  The  Commanders  drew  on  their  previous  experience  to  recognise  a  typical  action  to 
take. 

•  They  did  not  have  to  find  an  optimal  solution,  merely  a  workable  one. 

•  Once  they  had  arrived  at  a  suitable  course  of  action,  they  would  mentally  simulate  it 
first,  to  check  that  it  would  work. 

Consequently,  Klein  et  al.  (1986)  developed  the  RPD  model.  The  RPD  model  posits  that 
experienced  decision-makers: 
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•  Usually  consider  a  workable  option  and  do  not  need  to  generate  a  large  set  of 
alternatives. 

•  Generate  and  evaluate  options  one  at  a  time,  instead  of  comparing  the  advantages  and 
disadvantages  of  all  options. 

•  Evaluate  an  option  by  imagining  the  outcome,  and  by  finding  ways  to  avoid  problems 
that  may  arise  from  its  implementation. 

•  Focus  on  assessing  the  situation  and  looking  for  familiar  cues. 

•  Emphasise  acting  quickly  and  not  prolonging  analysis. 

The  RPD  model  describes  how  people  make  decisions  without  comparing  outcomes  (see 
Figure  1).  The  decision-maker  initially  assesses  the  situation,  looking  for  familiar  patterns 
or  prototypes.  This  allows  him/her  to  know  which  goals  make  sense,  what  relevant  cues 
to  expect  and  what  action  should  be  appropriate.  A  series  of  options  is  then  generated.  The 
first  solution  may  not  be  optimal,  but  it  will  usually  be  workable. 

This  model  also  shows  that  action  can  be  taken  quickly,  which  is  important  in  crisis 
management  (Klein,  1997).  The  RPD  model  has  been  applied  to  various  NDM  settings  and 
accepted  by  a  number  of  research  groups  as  providing  a  viable  description  of  cognitive 
processes  underlying  NDM  (Kaempf,  Wolf,  Thordsen,  &  Klein,  1996;  Mosier,  1991; 
Pascual  &  Henderson,  1997).  The  RPD  model  has  been  found  to  describe  decision-making 
in  domains  such  as  fireground  command,  wildland  fire  incident  command  teams,  US 
Army  armoured  division  personnel  (Klein,  1989),  battle  planning  (Thordsen,  Galushka, 
Young  &  Klein,  1987),  critical  care  nursing  (Crandall  &  Calderwood,  1989),  and  chess 
tournament  play  (Calderwood,  Klein  &  Crandell,  1988). 

To  understand  the  differences  between  the  environments  associated  with  the  RPD  model 
and  rational  choice  model  it  is  useful  to  contrast  the  environments  in  which  they  are 
applicable  (see  Figure  2). 

At  a  cognitive  level  it  has  been  suggested  that  the  RPD  model  incorporates  two  cognitive 
processes.  These  are: 

•  Situation  assessment  (which  is  used  to  generate  a  course  of  action) 

•  Mental  simulation  (which  is  used  to  evaluate  a  course  of  action) 
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Figure  1.  The  RPD  Model  (adapted  from  Klein,  1998) 
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Figure  2.  Comparing  the  RPD  Model  with  Classical  Decision-Making  (Klein,  1993a). 

3.3.4  Situation  Assessment  and  Situation  Awareness 

Situation  assessment  is  the  process  by  which  a  decision-maker  gathers  information 
pertaining  to  the  current  decision  problem.  Situation  assessment  occurs  initially  in  the 
RPD  model,  and  often  results  in  the  recognition  of  a  situation  from  past  experience.  The 
rest  of  the  decision-making  process  depends  on  correct  judgement  of  the  situation. 

It  has  been  argued  that  in  real-world  situations  it  is  uncommon  to  be  making  decisions  in  a 
static  environment  (Klein,  1998).  Thus,  in  contrast  with  the  singular  moments  of  choice,  as 
described  in  analytical  decision-making  models,  NDM  often  requires  a  series  of  decisions 
that  are  interdependent.  Therefore,  instead  of  relying  on  static  information,  the  decision¬ 
maker  must  constantly  monitor  the  changing  environment  to  inform  his/her  decision¬ 
making.  This  will  be  changing  spontaneously,  as  well  as  changing  because  of  each  decision 
made.  Thus,  NDM  requires  continual  situation  assessment.  Continual  assessment 
produces  situation  awareness  (SA)4. 


4  Situation  awareness  is  defined  as  " . .  .the  perception  of  the  elements  of  the  environment  within  a 
volume  of  time  and  space,  the  comprehension  of  their  meaning  and  the  projection  of  their  status 
into  the  near  future"  (Endsley,  1988,  p.  97). 


13 


DSTO-GD-0429 


Different  methods  have  been  suggested  as  explanations  for  how  decision-makers  arrive  at 
SA.  George,  Kaempf,  Klein,  Thorsden  and  Wolf  (1996)  found  that  military  officers 
identified  the  following  methods  of  arriving  at  SA: 

1.  87%  of  participants  used  a  feature  matching  strategy.  This  is  where  the  decision-maker 
sees  the  situation  as  familiar,  and  arrives  at  situation  awareness  through  a  series  of 
recognised  cues. 

2.  12%  of  participants  used  story  generation.  When  the  environment  does  not  provide 
enough  information  to  be  recognised  as  familiar,  the  decision-maker  constructs  a  story 
to  explain  the  information  and  to  arrive  at  greater  situation  awareness. 

3.  Only  1%  of  cases  did  not  fit  into  either  of  these  main  categories. 

Due  to  the  inclusion  of  SA  in  the  RPD  model,  a  theoretical  explanation  is  offered  for  the 
mechanisms  by  which  naturalistic  decision-makers  can  collect  and  make  sense  of  dynamic 
information.  Thus,  in  contrast  to  analytical  decision  models,  it  enables  the  RPD  model  to 
be  usefully  applied  to  describe  decision-making  in  dynamic,  fast  paced  situations. 

Research  in  the  field  has  identified  applications  of  SA  research  to  be  in  the  areas  of  training 
and  the  development  of  technology  and  policy. 

3.3.5  Mental  Models  and  the  Role  of  Mental  Simulation 

3.3. 5.1  The  Schema  and  Mental  Models  in  RPD  Theory  of  Expertise 

Lipshitz  and  Shaul  (1997)  argued  that  the  RPD  model  oversimplified  the  processes 
occurring  in  NDM.  They  suggested  that  this  was  evidenced  in  the  failure  of  the  model  to 
provide  adequate  explanations  their  research  finding,  which  is  now  outlined.  Their  study 
focused  on  the  differences  between  experts  and  novices  in  decision-making  in  a  sea- 
combat  simulation.  Lipshitz  and  Shaul  (1997)  found  that: 

•  Experts  collect  more  information  on  the  situation  before  they  make  a  decision  because 
they  know  what  questions  to  ask. 

•  Experts  engage  in  a  more  efficient  information  search  because  they  know  what 
information  to  keep  track  of. 

•  Experts  read  the  situation  more  accurately  because  they  are  able  to  identify  the 
difference  between  relevant  and  surplus  information. 

•  Experts  make  fewer  bad  decisions  related  to  reading  the  situation  correctly. 

•  Experts  communicate  more  frequently  and  elaborately  with  friendly  units. 

•  Experts  are  more  likely  to  consider  other  people's  perspectives  when  making  their 
decisions. 

Lipshitz  and  Shaul  (1997)  found  that  Klein's  RPD  model  accounted  for  most  of  these 
observations.  However,  it  could  not  account  for  experts  collecting  more  information  than 
novices  on  the  situation  before  they  make  a  decision.  As  described,  the  RPD  model 
specifies  those  conditions  under  which  a  decision-maker  would  collect  additional 
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information.  These  are:  when  the  decision-maker  does  not  recognise  the  situation  (thus 
cannot  establish  goals);  if  expectancies  are  violated;  or  the  expert's  mentally  simulated 
course  of  action  fails. 

If  Lipshitz  and  Shaul  (1997)  are  correct  however,  and  experts  always  collect  more 
information  than  novices,  it  could  be  inferred  that  in  comparison  with  novices: 

•  Experts  require  more  information  to  identify  goals. 

•  Experts  expectations  are  more  likely  to  be  violated. 

•  Experts  engage  in  mental  simulation  more  often. 

According  to  the  RPD  model,  the  third  implication  is  accurate  and  empirically  supported 
(Calderwood,  Crandall  &  Klein,  1987).  However  the  first  two  inferences  contradict  the  idea 
of  experts  being  efficient  decision-makers. 

Lipshitz  and  Shaul  (1997)  suggested  that  in  order  for  the  RPD  model  to  more  adequately 
explain  these  findings,  it  should  be  modified  to  include  schema  and  mental  models.  This 
would  account  for  experts  collecting  more  information,  and  conducting  more  effective 
searching.  Figure  3  represents  the  revised  model.  The  inclusion  of  these  ideas  offers  an 
explanation  of  how  an  expert  evaluates  a  course  of  action. 

In  order  to  facilitate  rapid  judgement  the  expert's  stored  schema5  is  retrieved  from 
memory.  A  schema  helps  determine  what  we  attend  to,  what  we  perceive,  and  what  we 
remember  and  infer.  Schema  also  help  to  specify  what  information  is  missing  and  where  it 
can  be  found.  Thus,  from  Figure  2,  continual  schema  guided  situation  assessment  enables 
the  expert  to  recognize  familiar  patterns.  The  inclusion  of  schema  in  a  description  of  expert 
decision-making  explains  why  experts  may  seek  more  information,  but  because  their 
information  seeking  is  guided,  actions  can  still  be  taken  quickly,  which  is  important  in 
crisis  management  (Klein,  1997).  This  may  also  explain  why  decision-making  can  seem 
intuitive  to  the  decision-maker. 

Situation  assessment  results  in  a  mental  representation  of  the  problem  at  hand.  Essentially 
it  is  a  "hypothetical  construct,  which  refers  to  the  operator's  ideas  about  a  system  and 
what  it  is  controlling"  (Mogford,  1990,  p.  4).  Thus,  based  on  current  information  and 
previous  knowledge  and  experience  the  decision-maker  forms  a  mental  model  of  the 
current  situation.  Following  this  the  mental  model  forms  the  lenses  through  which  the 
decision-maker  views  the  problem.  It  helps  to  shape  goal  building,  and  determine 
expectations  formed  (Wickens,  1984).  It  acts  as  a  mental  representation  of  the  perceived 
situation,  and  facilitates  consequent  judgements  and  decisions.  Mental  models  based  on 
good  situation  awareness  also  allow  decision-makers  to  project  the  environment's  status 
into  the  future  (Artman,  1998;  Brehmer,  1990).  This  is  referred  to  as  mental  simulation. 
This  process  is  depicted  in  a  model  adapted  from  Endsley  (1995)  in  Figure  4. 


5  A  schema  is  described  by  Vaughan  and  Hogg  (1995)  as  "  a  cognitive  structure  that  represents 
knowledge  about  a  concept  or  type  of  stimulus,  including  its  attributes  and  the  relations  among 
those  attributes."  (p.  31). 
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Figure  4.  The  role  of  mental  models  in  processing  current  situational  information  (adapted  from 
Endsley,  1995) 


33.5.2  How  Previous  Experience  is  Stored  in  Memory 

From  Section  3. 3.5.1  it  seems  apparent  that  previous  experience  is  an  integral  part  of  the 
recent  NDM  models.  A  cognitive  model,  proposed  by  Noble  (1993),  represents  how 
previously  solved  problems  are  stored  in  memory.  These  are  termed  "reference 
problems".  Each  reference  problem  can  be  broken  down  into  three  constituent  features 
(See  Table  2). 
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Table  2.  A  description  of  the  features  described  in  Noble's  (1993)  model  of  how  experience  is  stored 
in  memory 


Question 

Answered 

Memory  Features 

Description  of  Feature 

What? 

Objective  features 

This  aspect  of  the  memory  characterizes  the 
problem  and  its  objectives.  It  is  concerned  with 
concrete  features  (specific  observable  events  or 
objects). 

How? 

Action  features 

These  identify  the  steps  that  are  involved  in 
solving  the  problem.  These  are  abstract  features 
that  help  generalise  the  problem  type  and 
solution  method. 

Where? 

Environmental 

features 

These  comprise  cues  that  were  present  in  the  past 
problem  scenario.  These  are  environmental  cues 
that  may  indicate  a  similar  or  related  problem. 

This  model  therefore  identifies  three  levels  of  abstraction  in  memory  storage  of  memory. 
Using  this  model,  previously  experienced  problems  can  be  compared,  at  conceptually 
different  levels  of  memory,  with  the  current  problem.  From  these  cognitive  comparisons,  a 
problem  solution  that  is  likely  to  work  may  emerge  (see  Figure  5). 


Level  1:  Concrete  Features 


Level  2:  Abstract  (Estimated)  Features 


Level  3:  Environmental  Features 


Figure  5.  A  cognitive  model  of  memory  for  past  problem  scenarios  (Noble,  1993) 
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During  situation  assessment,  the  reference  problems  that  match  the  current  situation  are 
activated.  The  "problem  solutions  associated  with  those  in  memory  become  candidate 
actions"  (Noble,  1993,  p.  305).  This  is  in  accordance  with  NDM  being  proceduralised; 
meaning  that  decision-making  is  directly  linked  to  action  and  the  conditions  of  its 
applicability  (Means,  Salas,  Crandell  &  Jacobs,  1993;  Rasmussen,  1983). 

Frequently,  a  decision-maker  is  forced  to  make  a  decision  on  the  basis  of  incomplete 
information.  In  such  circumstances,  the  abstract  level  of  memory  facilitates  decision¬ 
making.  Noble  (1993)  concluded  that  in  such  instances  cognition  may  involve  the  use  of 
sophisticated  information  processing  on  general  world  knowledge.  This  cognitive  model 
allows  recognition-based  solutions  of  new  problems  that  resemble  but  are  not  necessarily 
identical  to  those  previously  experienced  (Cohen,  Freeman  &  Wolf,  1996). 

Studies  that  support  these  assumptions  include  George  et  al.,  (1996)  who  found  that 
military  officers  who  had  an  awareness  of  the  situation  were  able  to  recognise  actions 
appropriate  for  the  current  situation  from  published  procedures,  and  past  experiences. 
Also,  Lipshitz  (1989)  found  that  Israeli  army  officers  seemed  to  make  decisions  by 
matching  situations  to  an  associated  action.  Means  et  al.  (1993)  have  also  confirmed  that 
the  nature  of  a  person's  experience,  and  consequently  the  patterns  recognised,  affect  the 
decisions  they  make. 

Thus  far  the  concepts  of  SA  and  mental  models  have  seemed  to  provide  good  descriptions 
of  the  processes  believed  to  be  employed  by  naturalistic  decision-makers,  but  validating 
this  experimentally  has  proven  difficult.  Although  it  is  arguable  that  at  present  they  offer 
the  best  available  explanation  of  NDM  behaviour,  these  models  still  require  further 
elaboration  and  validation. 

3.3.6  Summarising  Descriptive  Decision-making  Theories 

In  summary,  descriptive  decision  theories  have  tried  to  describe  how  people  in  the  real 
world  actually  make  decisions.  Researchers  have  recognised  that  environments  such  as  the 
military,  fire  fighting,  and  emergency  services  present  a  specific  set  of  conditions  relevant 
to  the  generation  of  appropriate  decision  theories.  These  environments  have  been 
described  as  naturalistic  environments.  Naturalistic  environments  are  complex  and 
dynamic,  with  time  pressure,  high  stakes,  and  uncertainty.  It  has  been  recognised  that 
these  specific  conditions  require  a  new  theory  of  decision-making,  which  has  been  termed 
naturalistic  decision-making.  Within  the  relatively  new  field  of  NDM  several  decision¬ 
making  models  have  been  presented  as  alternatives  to  analytical  models.  One  example  is 
image  theory.  This  has  been  widely  cited  as  a  model  for  many  organisational  type 
decisions,  but  is  difficult  to  validate  and  has  not  been  successfully  applied  to  domains  such 
as  the  military.  Application  to  other  environments,  and  real-world  decision-makers,  still 
needs  to  be  achieved.  The  most  commonly  cited  model  proposed  to  describe  decision¬ 
making  in  naturalistic  environments  is  the  RPD  model.  This  has  been  adapted  to  include 
schema  and  mental  models,  and  has  formed  the  platform  for  recent  NDM  research.  In 
addition  to  this  model.  Noble  (1993)  has  created  a  cognitive  model  of  how  experience  is 
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stored  in  memory  for  use  in  NDM.  This  model  describes  how  previous  experience  feeds 
into  decision-making,  and  how  incomplete  information  is  handled.  Overall,  these  models 
still  require  further  elaboration  and  validation,  but  they  do  offer  important  theoretical 
ideas  such  as:  situation  awareness,  the  use  of  previous  experience,  schema  and  mental 
models. 


4.  Previous  Experience  and  Expert  Functioning 

4.1  Defining  Expertise 

A  critical  component  of  decision-making  in  naturalistic  settings  is  specialised  domain 
knowledge,  which  is  a  product  of  experience.  Despite  its  inextricable  link  with  the  RPD 
model,  expertise  has  not  been  adequately  defined  (Shanteau,  1992).  There  seems  to  be  a 
lack  of  agreement  within  the  literature,  with  definitions  varying  from  study  to  study 
(Shanteau,  1992).  Many  studies  use  hours  of  experience  as  an  indicator  (Klein  & 
Calderwood,  1986;  Klein  et  al.,  1986).  A  repertoire  of  10  years  experience  is  generally 
accepted  to  qualify  a  person  as  an  expert  (Klein  &  Calderwood,  1986).  However,  it  is  not 
only  length  of  experience  that  is  important;  quality  of  experience  should  also  be 
considered.  For  example,  Klein  et  al.  (1986)  found  that  for  firefighters,  10  years  on-the-job 
experience  was  not  as  valuable  for  skill  development  as  one  year  in  a  decaying  inner  city. 
Shanteau  (1992)  suggests  that  in  order  to  define  expertise  we  need  to  procure  the 
assistance  of  those  in  the  field  to  identify  people  who  have  the  necessary  skills  and  abilities 
to  perform  at  the  highest  level  within  their  specialised  domain. 

Because  of  the  difficulty  in  accurately  defining  expertise,  research  has  tended  to  focus  on 
the  comparison  of  experts  and  novices  in  decision-making.  Thus,  an  understanding  of 
expertise  is  mediated  by  novice  performance. 

To  provide  an  overview  of  research  in  this  area,  Glaser  and  Chi  (1988)  reviewed  the 
literature  and  extracted  a  list  of  themes  that  have  become  commonly  associated  with 
expert  performance. 

•  Expertise  is  domain  specific. 

•  Experts  see  patterns. 

•  Experts  are  faster  and  make  fewer  errors. 

•  Experts  have  superior  memory  in  their  domain  (environmental  cues  are  an  aid  to 
recall). 

•  Experts  see  and  represent  a  problem  at  a  deeper  level. 

•  Experts  spend  more  time  trying  to  understand  a  problem  but  have  a  more  efficient  path 
to  solving  it. 

•  Experts  have  strong  self-monitoring  skills. 

•  Experts  have  refined  perceptual  abilities. 
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4.2  Linking  Expertise  with  Naturalistic  Decision-making 

Experience  has  been  recognised  as  important  in  many  instances.  For  example,  in  a 
summary  of  the  great  military  blunders  throughout  history,  David  (1997)  noted  that  a 
"typical  handicap  seems  to  be  a  lack  of  command  experience.  Naivety  tends  to  promote 
vacillation  and  overcaution,  resulting  in  lost  opportunities  and  ultimately  disaster"  (p.  1). 
In  this  case,  the  application  of  expert  knowledge  to  NDM  is  targeted  as  the  factor 
differentiating  between  good  and  poor  decision  outcomes.  In  reality  the  relationship  may 
not  be  quite  as  clear.  It  is  likely  that  external  factors  also  mediate  decision  outcomes 
(Woods,  Johannesen,  Cook,  &  Sarter,  1993).  However  it  has  been  shown  repeatedly  that 
experts  tend  to  have  different  perceptual  and  cognitive  mechanisms  in  comparison  with 
novices,  which  facilitate  their  decision-making  (eg.  Klein  &  Hoffman,  1993;  Randel,  Pugh 
&  Reed,  1996).  These  will  be  elaborated. 

NDM  theory  suggests  that  experts  draw  on  previous  experience  to  make  sense  of  current 
situations.  They  rely  on  familiar  stimulus  cues  in  order  to  predict  the  success  of  a  decision 
solution.  It  has  been  suggested  that,  with  experience,  people  learn  which  cues  to  attend  to, 
and  what  information  to  filter  out  (George  et  al.,  1996.).  Therefore  it  would  be  expected 
that  a  broader  range  of  experience  would  provide  more  recognitional  cues  upon  which  the 
decision-maker  may  draw  when  making  time-pressured  decisions.  There  has  been 
extensive  research  on  the  types  of  cognitive  and  perceptual  skills  that  are  linked  to  expert 
performance.  On  the  other  hand,  the  substance  of  the  experience  that  experts  have 
acquired  has  been  left  largely  unexplored.  For  example,  the  context  of  the  experience  and 
the  mode  of  acquisition  (eg.  Direct  experience,  knowledge  gained  from  lessons,  informal 
communication  etc)  have  not  been  investigated.  This  presents  limitations  in  defining 
expertise  and  thus  impacts  on  the  usefulness  of  the  RPD  model  for  training. 

4.3  Perceptual-Cognitive  Aspects  of  Expertise 

There  has  been  debate  in  the  literature  over  whether  experts  actually  use  different,  more 
refined  higher-level  skills  (such  as  methods  of  problem  solving),  or  whether  the 
expert/ novice  distinction  is  perceptual.  Means  and  Gott  (1988)  suggested  that  expert  and 
less  expert  decision-makers  apply  the  same  rules  but  to  a  different  content.  Beach  and 
Lipshitz  (1997)  extended  this  idea  by  stating  that  decision-making  events  require  decision¬ 
makers  to  use  their  existing  knowledge  of  the  context  to  facilitate  both  their  situation 
assessment  and  subsequent  decision-making.  According  to  the  RPD  model,  it  seems  that 
both  perceptual  and  cognitive  skills  distinguish  expert  decision-makers  from  novices. 
These  skills  will  be  discussed  in  terms  of  their  contribution  to  the  RPD  model. 

4.3.1  Expert  Information  Representation 

Klein  and  Hoffman  (1993)  argued  that  experts  use  their  knowledge  in  order  to  gain  a 
perceptual  advantage.  They  stated  that  "expertise  is  a  function  of  the  knowledge  base 
itself,  and  that  as  people  develop  richer  knowledge  bases  they  are  able  to  represent 
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problems  in  more  powerful  ways"(p.  208).  In  other  words,  experts  are  able  to  see  what  is 
"invisible"  to  novices,  such  as  situation  typicality,  the  ability  to  see  distinctions  and  to 
identify  antecedents  and  consequences.  Therefore  the  knowledge  one  accumulates  is  not 
as  important  as  how  one  sees  the  world  (Hutton  &  Klein,  1999).  This  perceptual  advantage 
of  experts  is  linked  to  a  number  of  specialized  techniques  believed  to  be  employed  by 
experts.  These  are  described  below. 

4.3.1.1  Attend  to  Relevant  Information 

Experts  establish  situation  awareness  by  relying  on  familiar  stimulus  cues.  It  has  been 
suggested  that,  with  experience,  people  learn  which  cues  to  attend  to,  and  what 
information  to  filter  out6  (George  et  al.,  1996).  This  allows  experts  to  process  relevant 
information  more  quickly,  and  not  to  experience  cognitive  overload.  Due  to  their  ability  to 
filter  information,  experts  are  able  to  take  into  account  more  relevant  information  than  a 
novice  is  able  to  (Federico,  1995;  Hutton  &  Klein,  1999).  Less  expert  decision-makers  may 
experience  cognitive  overload  more  rapidly  because  they  attend  to  all  information  and  do 
not  filter  irrelevant  aspects  (Randel  et  al.,  1996). 

4.3.1. 2  Chunking 

The  classic  study  by  DeGroot  (1965/1978),  later  replicated  by  Chase  and  Simon  (1973), 
compared  the  performance  of  chess  masters  and  novices.  The  researchers  were  interested 
in  isolating  factors  that  separate  an  expert  from  a  novice.  They  found  that  experts  and 
novices  did  not  differ  significantly  on  memory  abilities,  depth  of  planning  or  other  similar 
areas.  Instead,  the  difference  was  that  the  expert  players  could  look  at  the  entire  complex 
chess  display  and  condense  it  into  "meaningful  chunks",  whereas  the  novices  tried  to 
understand  the  whole  display  (Means  et  al.,  1993,  p.  310).  This  ability  also  seems  to  be 
consistent  with  the  idea  of  schema  driven  information  search. 

4.3.1.3  Expert  Knowledge  Base 

Experts  have  greater  domain  knowledge  than  novices.  It  has  been  found  that  their 
knowledge  is  also  organised  differently.  Novices  will  organize  representations  around 
surface  features,  whereas  experts  will  use  inferences  and  principles  (Chi  &  Glaser,  1981). 
Glaser  (1987)  suggested  that  the  quality  of  representation  of  a  particular  problem  depends 
on  the  relations  between  the  structure  of  a  knowledge  base  and  problem  solving  processes 
used.  The  decision-maker  constructs  a  mental  model7  on  the  basis  of  domain-related 
knowledge,  and  how  this  knowledge  is  organized.  Therefore,  the  content  and  organization 
of  domain  knowledge  determines  the  coherence  of  the  representation.  This  would  then 
determine  the  quality  of  decision  being  made.  This  idea  is  consistent  with  the  idea  of 
mental  models  within  the  RPD  model. 


6  Refer  to  section  3.3.5. 1,  which  outlines  the  inclusion  of  schema  in  the  RPD  model. 

7  Refer  to  section  3.3.5. 1,  which  outlines  the  inclusion  of  mental  models  in  the  RPD  model. 
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4. 3.1.4  Pattern  Matching 

Experts  can  see  typicality  by  using  pattern-matching  techniques.  When  experiencing  an 
event,  information  about  that  event  is  stored  in  memory.  When  looking  at  a  current  event, 
patterns  can  be  recognised  as  similar  to  those  previously  stored.  Simon  &  Gilmartin  (1973) 
estimated  that  chess  masters  store  10,000-100,000  patterns  in  their  memory.  Chi, 
Hutchinson  and  Robin  (1988)  also  supported  this  view.  They  studied  children  who  were 
dinosaur  "experts"  and  "novices" .  These  children  were  presented  with  a  series  of  dinosaur 
pictures  and  asked  to  identify  whether  or  not  it  was  a  dinosaur.  It  was  found  that  the 
"novices"  failed  to  judge  typicality.  This  idea  is  consistent  with  Noble's  (1993)  model  of 
knowledge  storage.  It  has  also  been  discussed  as  being  a  critical  tool  for  generating  a 
typical  course  of  action  through  situation  recognition  (Klein,  1998). 

4.3.2  Cognitive  Strategies 

4.3.2.1  Problem  Solving  Method 

It  has  been  suggested  by  some  researchers  that  experts  and  novices  do  not  differ  in  their 
ability  to  use  particular  problem  solving  methods  or  decision  rules,  but  rather  in  how  they 
use  their  domain  knowledge  within  this  process  (Means  et  al.,  1993;  Drillings  &  Serfaty, 
1997;  Chi  &  Glaser,  1981).  Studies  that  have  supported  this  idea  include  Chi  and  Glaser 
(1981),  who  found  that  both  novices  and  experts  can  use  both  bottom  up8  and  top  down9 
reasoning  strategies.  In  contrast  Mullin  (1989)  found  that  experts  use  bottom  up  rather 
than  top  down  processes.  The  type  of  information  the  participants  were  supplied  with 
may  explain  the  contradictory  nature  of  these  findings.  In  the  study  by  Mullin  (1989) 
novices  may  have  been  forced  to  use  only  using  top  down  processing  because  of  their 
limited  knowledge  base  in  the  field.  Thus,  it  seems  that,  although  strategies  do  not 
determine  expertise,  it  may  be  that  people  make  use  of  different  strategies  to  complement 
and  take  advantage  of  their  declarative  knowledge,  perceptual  and  cognitive  knowledge 
and  other  aspects  of  their  experience  base  (Klein,  1993a,  1993b). 

4.3.22  Mental  Simulation 

Experts  are  also  better  at  dealing  with  problems  that  have  not  been  previously 
encountered.  This  can  be  explained  by  understanding  their  method  of  progressive 
deepening  or  mental  simulation.  Once  an  option  is  considered,  experts  will  use  mental 
simulation  to  work  it  through  at  a  deeper  level,  looking  for  pitfalls  and  opportunities. 
Mental  simulation  allows  experts  to  project  the  environment's  status  into  the  future 
(Artman,  1998).  This  suggests  that  experts  and  novices  may  have  the  same  problem 


8  Bottom-up  processing-  Based  on  the  situation,  particular  solution  methods  seem  appropriate. 
Individuals  work  through  the  problem  and  generate  expected  outcomes.  This  type  of  processing  is 
automatic  and  does  not  require  metacognition. 

9  Top-down  processing-  the  context  guides  the  cognitive  processes  needed  to  determine  whether 
problem  expectations  are  being  met.  This  type  of  processing  is  high  level,  slow  and  requires  active 
commitment  and  cognitive  capacity. 
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solving  strategies  available,  but  experts  can  use  them  more  effectively  because  of  their 
superior  knowledge  base  and  perceptual  advantage.  They  are  able  to  categorise  problems 
according  to  their  large  repertoire  of  knowledge-based  schema,  and  then  apply  basic 
problem  solving  techniques. 

Thus  there  has  been  a  range  of  skills  associated  with  expert  decision-making  performance, 
and  these  have  been  successfully  incorporated  into  the  RPD  model.  Further  research  still 
needs  to  be  done  in  order  to  produce  an  adequate  definition  for  expertise  that  can  be 
applied  in  the  future  development  of  expert  training  systems.  For  example  it  is  important 
to  clarify  exactly  what  is  encompassed  within  the  terms  like  'previous  experience  or 
specialised  domain  knowledge,  because  these  are  not  yet  clearly  definable. 

4.4  Summarising  Expertise  in  Naturalistic  Decision  Making 

Expertise  is  inextricably  linked  with  the  concept  of  NDM  (Klein,  1998).  Research  has 
tended  to  focus  on  the  comparison  of  experts  and  novices  in  decision-making.  Thus,  an 
understanding  of  expertise  is  typically  mediated  by  novice  performance.  A  review  of 
research  on  expertise  outlined  that  number  of  facets  of  expertise  were  common  across 
domains  (Glaser  &  Chi,  1988).  Flowever,  to  date  an  adequate  definition  of  expertise  has  not 
been  generated.  This  needs  to  be  addressed  in  future  research.  Researchers  have  also 
debated  whether  expertise  is  due  to  the  development  of  specialised  perceptual  or  cognitive 
skills.  As  has  been  shown,  the  current  RPD  model  successfully  incorporates  both  types  of 
skills  in  its  description  of  decision-making.  However,  it  still  remains  problematic  as  to  the 
exact  nature  of  expertise,  and  future  research  is  required.  Establishing  a  good 
understanding  of  this  at  the  cognitive  level  allows  for  more  intuitive  interface  design  for 
expert  decision-makers,  because  specific  information  requirements,  and  appropriate 
modes  of  representation  can  be  ascertained. 


5.  Levels  of  Expertise 

Research  has  tended  to  focus  on  expert/ novice  comparisons,  mainly  due  to  the  lack  of  a 
clear  definition  of  expertise  within  the  literature.  One  method  of  generating  a  greater 
understanding  of  expert/  novice  differences  is  by  seeking  to  understand  cognitive  skill 
development. 

Expertise  in  decision-making  can  be  considered  as  a  skill  that  can  be  developed.  Skills  can 
be  broken  down  into  two  categories:  cognitive  and  motor  skills.  Expert  decision-making  in 
NDM  fits  into  the  first  category,  but  realistically  decision  outcomes  also  often  rely  on 
motor  skills.  As  classified  by  Sabol  and  Wisher  (2001)  military  tasks  comprise  three 
components: 
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•  Knowledge-  based  on  recall  of  domain  specific  information.  This  depends  on 
experience. 

•  Decision-  depends  on  cognitive  processing  of  the  domain-specific  information. 

•  Execution-  refers  to  the  perceptual  motor  requirements  of  a  task,  for  example,  target 
acquisition  and  tracking. 

Therefore,  expert  decision-making  outcomes  are  dependent  on  these  three  things.  NDM 
theory  tends  to  consider  both  the  knowledge  of  the  decision-maker  and  the  decision 
process  within  the  RPD  model,  but  does  not  include  the  decision  execution.  The  literature 
indicates  that  there  is  a  link  between  cognitive  skill  (knowledge  and  decision)  and 
performance  (motor  skill;  Welford,  1968).  One  could  make  a  great  decision,  but  implement 
it  poorly,  and  the  performance  outcome  would  therefore  be  poor.  In  a  setting  such  as  a 
military  HQ,  because  the  quality  of  a  decision  is  generally  determined  by  the  quality  of  the 
decision  outcome,  it  needs  to  be  recognised  that  this  is  not  only  dependent  on  the  key 
decision-maker.  Instead  consideration  should  also  be  given  to  the  quality  of  the 
communication  between  team  members,  and  the  quality  of  motor  skills  involved  in  task 
execution  and  other  contextual  factors. 

For  the  purpose  of  this  section,  developing  expertise  in  NDM  will  be  treated  as 
synonymous  with  cognitive  skill  development.  From  a  literature  review  it  seems  that  there 
is  overlap  between  skill  development  and  more  recent  articles  on  development  of 
expertise  in  NDM.  However,  it  is  noted  that  development  of  cognitive  skills  does  not 
necessarily  equate  to  good  decision  outcomes,  as  other  factors  may  impact  on  this. 

5.1  Development  of  Expertise:  Skill  Acquisition 

As  expertise  increases,  individuals  use  tacit  knowledge,  and  perceive  the  world  through 
categories.  This  is  referred  to  as  the  proceduralisation  of  declarative  knowledge  (Means  et 
al.,  1993).  The  understanding  of  how  a  skill  is  developed  requires  an  understanding  of 
how  it  is  committed  to  memory.  Welford  (1968)  outlined  the  basic  principles  involved  in 
learning  a  skill  (see  Figure  6): 

1.  Understanding  the  task  (material  must  be  perceived  and  comprehended). 

2.  Material  held  in  short  term  memory  (STM)  until  there  is  time  for  more  permanent 
registration  to  take  place. 

3.  A  memory  trace  will  be  established.  This  must  be  resilient  to  subsequent  activities 
interference. 

4.  The  memory  trace  must  endure  till  time  of  recall. 

5.  Recognition  of  an  appropriate  situation  in  which  to  use  information  must  occur. 

6.  Material  must  be  recovered  correctly  ("tip  of  the  tongue"  phenomena). 

7.  Recalled  material  should  be  used  to  produce  a  communicable  response. 
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Figure  6.  The  storage  of  experience  in  memory  (adapted  from  Welford,  1956) 

In  order  for  a  skill  to  be  learnt,  all  of  these  stages  must  take  place,  and  problems  occurring 
at  any  of  these  stages  can  hinder  the  process. 

Once  the  basic  process  of  skill  acquisition  is  understood,  it  can  then  be  applied  to  a  given 
domain  (e.g.  Military).  With  experience,  performance  of  skills  has  been  noted  to  change. 
Different  stages  of  skill  have  been  identified  (Anderson,  1991;  Fitts  and  Potter  1967;  cited 
in  Shanteau,  1992).  These  are:  cognitive  stage  (fact  is  committed  to  memory  via  rehearsal), 
associative  stage  (links  are  made  between  facts),  automatic  stage  (links  are  smooth  and 
continuous  and  less  conscious  attention  is  needed  to  perform  the  skill). 

These  stages  are  similar  to  those  proposed  by  Rassmussen  (1983),  who  describes  the 
development  of  expertise  by  breaking  it  down  into  three  different  phases: 

1)  Knowledge-based  behaviour  -  Requires  conscious  attention  that  involves  a  higher 
level  of  cognitive  processing  or  thought  than  does  rule  based  control.  "Know-how" 
and  control  rules  are  not  available.  The  relevant  knowledge  that  the  decision-maker 
does  have  still  needs  to  be  proceduralised.  Multiple  courses  of  action  are  often 
developed  and  tested  against  the  goal. 

2)  Rule  based-behaviour  -  Behaviour  is  directed  by  a  sub  routine  or  procedure  that 
consists  of  a  series  of  rules.  It  is  goal  oriented.  The  decision-maker  is  aware  that 
alternative  actions  are  possible  to  reach  the  goal  and  that  a  choice  amongst  them  can 
be  made. 

3)  Skill-based  behaviour  -  Not  requiring  conscious  attention.  Attuned  to  feedback  and 
easily  adjustable. 
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Probably  the  most  cited  model  of  expert  performance  is  that  by  Dreyfus  (1972).  This  model 
provides  five  stages  that  describe  the  development  of  expertise:  novice,  advanced 
beginner,  competent,  proficient,  and  expert.  As  someone  develops  expertise  and  moves 
through  the  stages  outlined  they  will  progress  in  much  the  same  way  as  described  by 
Rassmussen  (1983).  The  progression  through  these  skill  sets  begins  with  variable  awkward 
performance  and  continues  to  a  point  where  decisions  become  intuitive,  and  responses 
more  immediate  and  logical  to  the  decision-maker  (Fitts  &  Potter  1967;  cited  in  Shanteau, 
1992). 

As  one  passes  through  these  stages  outlined  by  Dreyfus  (1972),  skill  development  is 
occurring  in  more  than  one  area.  Glaser  (1987)  isolates  the  four  skill  areas  that  develop  as 
one  acquires  expertise  (see  Table  3). 


Table  3.  Key  areas  of  change  as  expertise  in  NDM  is  attained  (Glaser,  1987) 


Skill  Area 

Type  of  Development 

Performance 

Variable,  awkward  performance  becomes  consistent,  accurate, 
complete  and  relatively  fast. 

Judgement 

Individual  acts  and  judgements  are  integrated  into  overall 
strategies. 

Perception 

Perceptual  learning  occurs  so  that  a  focus  on  isolated  variables 
shifts  to  the  perception  of  complex  patterns. 

Locus  of  control 

Increased  self-reliance  and  the  ability  to  form  new  strategies 
when  required. 

As  can  be  seen,  as  a  person  develops  their  cognitive  skill  (such  as  decision-making), 
processing  becomes  more  effective  and  efficient.  As  one  reaches  expertise,  often  cognitive 
processing  is  not  occurring  at  a  conscious  level.  The  term  'intuition'  has  been  quite 
popular  in  recent  literature  to  describe  this  type  of  fluid  information  processing  and 
decision-making  (see  Klein,  2003). 

This  information  on  cognitive  skill  development  is  important  when  studying  decision¬ 
makers  as  it  provides  an  understanding  of  how  they  are  using  and  developing  their  skills 
as  they  perform  decision-making.  It  also  gives  us  another  means  of  distinguishing 
differences  between  expert  and  novice  decision-makers  for  input  to  training  technologies. 


5.2  Summarising  Development  of  Expertise 

In  summary,  understanding  the  development  of  expertise  is  essential  in  order  to  more 
fully  comprehend  the  expert/ novice  continuum.  Developing  expertise  in  NDM  can  be 
seen  as  synonymous  with  acquiring  a  cognitive  skill.  By  considering  literature  on  skill 
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development,  and  the  role  of  memory,  we  are  able  to  understand  how  general  cognitive 
principles  can  apply  to  understanding  expert  performance.  Skill  acquisition  can  initially  be 
explored  at  a  cognitive  level.  The  process  of  learning  and  storage  of  skills  in  long-term 
memory  represents  a  basic  platform  on  which  to  base  further  understanding  of  expertise 
(summarised  by  Welford  [1968]).  From  there,  we  are  able  to  describe  how  expertise 
develops  as  experience  in  a  domain  increases.  The  development  of  expertise  has  been 
described  in  stages  by  Anderson  (1991),  Dreyfus  (1972),  and  Rasmussen  (1983).  These 
models  describe  decision  skills  as  beginning  as  variable  and  awkward  and  progressing  to 
a  point  where  decisions  become  intuitive,  and  responses  more  immediate  and  logical  to 
the  decision-maker.  The  skills  that  are  developed  during  the  translation  from  novice  to 
expert,  have  been  broken  down  into  four  skill  areas  (Glaser,  1987).  These  are  performance, 
judgement,  perception  and  locus  of  control.  This  information  is  important  for  improved 
understanding  of  the  distinction  between  expert  and  novice  decision-makers,  and  also  to 
input  into  future  training  technologies. 


6.  Teamwork  and  Naturalistic  Decision  Making 

Many  organisations  require  teams  of  decision-makers  in  order  to  handle  the  complexity  of 
the  tasks  performed.  This  is  certainly  the  case  with  the  Australian  Army,  where  HQ  teams 
are  established  in  order  to  achieve  complex  planning  and  decision-making  duties.  Within 
this  team,  the  commander  is  ultimately  responsible  for  the  decisions  made,  but  the  team 
guides  the  decision-making  process.  Therefore,  it  is  important  to  understand  how  teams 
can  impact  on  the  commander's  decision-making  process. 

6.1  Group  or  Team? 

In  the  psychological  literature  there  has  been  debate  surrounding  the  difference  between 
teams  and  groups.  Group  has  been  used  as  a  much  more  broad  descriptor  than  team. 
Johnson  and  Johnson  (1987)  define  a  group  as  "  two  or  more  individuals  in  face-to-face 
interaction,  each  aware  of  his  or  her  group  membership,  each  aware  of  the  others  who 
belong  to  the  group,  and  each  aware  of  their  positive  interdependence  as  they  strive  to 
achieve  mutual  goals"  (p.  8).  It  has  been  argued  that  a  team  is  more  structured,  in  that  each 
member  has  a  role  and  works  interdependently  toward  a  common  meaningful  goal 
(Morgan,  Glickman,  Woodard,  Blaiwes  &  Salas,  1986).  Teams  require  co-ordination,  which 
results  in  simultaneity  or  sequencing  in  order  to  coordinate  their  efforts  (Brannick  & 
Prince,  1997).  Teams  also  tend  to  have  a  history  and  a  future,  where  groups  will  often  be 
formed  and  disbanded  to  achieve  a  specific  short-term  task.  Paris,  Salas  and  Cannon- 
Bowers  (2000)  identify  other  characteristics  that  delineate  teams  from  small  groups  as: 
multiple  sources  of  information,  task  interdependencies,  co-ordination  amongst  members, 
common  and  valued  goals,  specialized  member  roles  and  responsibilities,  task  relevant 
knowledge,  intensive  communication,  and  adaptive  strategies  to  help  respond  to  change. 
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If  we  are  to  take  the  definition  one  step  further  and  look  at  differences  between  group 
decision-making  and  team  decision-making,  we  can  focus  on  task  instead  of  simply 
structure.  It  has  been  suggested  that  in  groups  the  decision-making  is  the  task,  whereas  for 
teams  the  decision-making  leads  to  the  task  (Jones  &  Roelofsma,  2000).  Looking  at 
command  and  control  teams  in  particular,  such  as  an  Army  HQ  team,  we  must  realise  that 
they  participate  in  both  conditions.  This  makes  the  use  of  this  element  of  the  definition 
rather  unhelpful. 

It  is  fair  to  conclude  from  this  summary  that  a  team  is  a  kind  of  a  group.  Therefore,  some 
of  the  same  principles  apply,  and  some  research  on  groups  could  be  equally  applicable  to 
a  team  setting.  The  reverse  however  is  not  true. 

6.2  Group  Properties  and  Performance 

Any  group  of  people  working  together  over  a  period  of  time  have  been  found  to  have 
some  basic  social  properties.  These  are: 

•  Cohesiveness  -  This  is  colloquially  known  as  team  spirit  or  morale.  It  is  believed  to  be 
the  psychological  process  that  transforms  many  individuals  into  a  group  (Vaughan  & 
Hogg,  1995).  Operationalising  the  concept  has  been  difficult,  but  it  seems  to  be  related 
to  interpersonal  attraction,  increased  conformity,  improved  inter-group  commun¬ 
ication,  and  liking. 

•  Socialisation  -  This  can  be  defined  as  the  "dynamic  relationship  between  the  group 
and  its  members  that  describes  the  passage  of  its  members  through  a  group  in  terms  of 
commitment  and  changing  roles"  (Vaughan  &  Hogg,  1995). 

•  Norms  -  These  are  shared  beliefs  about  the  appropriate  behaviour  for  a  member  of  the 
group.  They  provide  a  frame  of  reference  for  group  members  to  choose  appropriate 
behaviour  for  themselves,  and  evaluate  the  behaviour  of  others. 

These  same  properties  should  be  found  in  a  HQ  team  in  the  Australian  Army,  because 
they  are  an  aggregate  of  people  that  work  together  on  a  day-to-day  basis.  Therefore,  these 
group  properties  would  be  exhibited. 

Two  structural  properties  that  are  commonly  identified  in  groups  are: 

•  Roles  -  These  represent  the  division  of  labour  in  a  group. 

•  Communication  network  -  This  can  be  defined  as  a  set  of  rules  that  govern 
appropriate  communication  between  group  members. 

It  seems  that  these  properties  are  also  both  consistent  with  the  Australian  Army.  Firstly,  in 
the  Army,  roles  are  explicitly  stated  (eg.  S3,  S2),  and  also  a  communication  network  seems 
to  be  present  which  is  influenced  by  the  Army  rank  system. 

Aside  from  understanding  properties  exhibited  by  groups,  research  has  also  focused  on 
the  effect  of  a  group  on  performance  outcomes.  Research  has  found  that  the  mere  presence 
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of  others  is  arousing.  This  presence  acts  to  facilitate  well-learned  responses  or  to  impede 
novel  tasks,  or  poorly  learnt  skills  (drive  theory;  Zajonc,  1965).  Since  then  it  has  been 
suggested  that  it  may  not  be  the  presence  of  others  that  affects  performance,  but  our 
understanding  of  social  rewards  and  punishments  (Cottrell,  1972). 

Another  phenomenon  observed  in  groups  is  transactive  memory.  An  advantage  of 
working  in  a  group  is  that  groups  tend  to  remember  more  information  than  the  best 
individual  in  that  group  (Clark  &  Stephenson,  1989).  The  term  transactive  memory  refers 
to  a  theory  developed  by  Wegner,  Erber  and  Raymond  (1991)  which  suggests  that  groups 
develop  a  shared  system  for  encoding  knowledge,  where  each  individual  is  only 
responsible  for  part  of  what  the  group  needs  to  know.  Each  member  understands  who  is 
responsible  for  each  memory  domain. 

6.3  Research  on  Teams 

Since  the  1950s,  team  research  has  regularly  evolved.  There  are  eight  categories  into  which 
theories  of  teamwork  have  developed  over  the  last  half-century.  These  are:  social- 
psychological  approach,  socio-technical  approach,  ecological  approach,  human  resource 
approach,  technological  approach,  lifestyle  approach,  functional  (taxonomic)  approach, 
and  the  integrative  approach  (for  a  summary  see  Paris  et  al.,  2000). 

In  addition,  researchers  have  worked  to  identify  factors  that  are  internal  and  external  to 
individuals  that  may  affect  team  decision-making.  Critical  skills  or  traits  that  may  be 
related  to  effective  teamwork  were  summarised  by  Cannon-Bowers,  Tannenbaum,  Salas 
and  Volpe  (1995)  and  Salas  and  Cannon-Bowers  (2000)  into  three  main  categories.  These 
were:  cognitions,  skills  and  attitudes.  Ideas  contained  within  these  three  broad  areas  are 
summarised  in  Table  4. 


Table  4.  A  summary  of  research  on  critical  components  of  teamwork  (Salas  &  Cannon-Bowers, 
2000) 


Characteristic 

Ideas  Encompassed 

Cognitions 

Cue-strategy  associations,  task  specific  team-mate  characteristics, 
shared  task  models,  team  mission,  objectives,  norms,  resources,  task 
sequencing,  accurate  task  models,  problem  models,  team  role 
interaction  patterns,  teamwork  skills,  boundary  spanning  roles  and 
team  orientations. 

Skills 

Adaptability,  shared  situation  awareness,  mutual  performance 
modelling,  motivating  team  members,  team  leadership,  mission 
analysis,  communication,  decision  making,  assertiveness,  inter¬ 
personal  co-ordination,  and  conflict  resolution. 

Attitudes 

Motivation,  collective  efficacy,  shared  vision,  team  cohesion,  mutual 
trust,  collective  orientation,  and  importance  of  teamwork. 
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Aside  from  this,  it  is  important  to  be  aware  of  the  external  variables  that  can  potentially 
contribute  to  team  performance  outcomes.  Table  5  shows  how  these  have  been 
summarized  by  Morgeson,  Aiman-Smith  and  Campion  (1997)  and  Meister  (1985). 

Table  5.  Description  of  variables  that  may  influence  team  performance  ( For  examples  see  Appendix  A) 


Factor(s)  Description 


Contextual  Variables  that  pertain  to  the  environment  in  which  the  team  activity  is 
Factors  embedded. 


Structural  Variables  impinging  primarily  from  sources  external  to  the  team,  but 
Factors  may  include  some  internal  to  the  team  (eg.  team  organisation). 

Team  Design  Variables  inherent  to  the  team  itself;  the  way  in  which  it  is  designed. 

Factors 


Process  Variables  inherent  to  the  team  itself;  the  way  in  which  it  functions. 

Factors 


Contingency  Variables  impinging  from  sources  internal  and  external  to  the  team. 

Factors 


6.4  Teams  and  Shared  Mental  Models 

In  the  last  decade,  a  substantial  amount  of  team  research  has  focused  on  the  concept  of 
shared  mental  models  (SMM).  Researchers  agree  that  teams  tend  to  develop  SMM 
(Cannon-Bowers,  Salas  &  Milano vich,  1999).  These  can  essentially  be  described  as 
knowledge  structures  or  cognitive  representations  that  are  shared  between  team  members, 
and  are  used  to  organise  information,  understand  events  and  make  predictions  and 
decisions  (Rouse  &  Morris,  1986).  They  allow  for  a  similar  understanding  of  the  situation 
amongst  team  members,  and  guide  how  team  members  will  interact  (Gentner  &  Stevens, 
1983).  Shared  mental  models  have  also  been  used  to  explain  the  co-ordinated  performance 
of  teams  (Stout,  Cannon-Bowers,  Salas  &  Milanovich,  2000).  In  particular,  it  has  been 
suggested  that  an  SMM  can  lead  to  the  use  of  more  proactive  communication.  This,  in 
turn,  has  been  linked  to  effective  performance.  For  example  it  was  found  that  effective 
teams  tend  to  provide  information  without  prompting  and  provide  information  in 
advance  (Stout  et  al.  2000). 

It  has  been  suggested  that  a  number  of  characteristics  can  contribute  to  the  development  of 
SMM.  For  example,  teams  that  engage  in  effective  planning  before  a  mission  (task)  appear 
to  develop  better  SMM  than  those  that  do  not  (Stout,  Cannon-Bowers  &  Salas,  1996). 
Following  from  this,  teams  that  planned  more  effectively  were  also  found  to  perform 
better  under  periods  of  increased  workload  (Stout  et  al.,  2000). 
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Klein  and  Thorsden  (1990)  put  forward  the  idea  of  the  "team  mind".  Essentially  taking  the 
idea  of  shared  mental  models  a  step  further,  they  propose  that  a  team  can  be  looked  at  as 
analogous  to  the  mind  of  one  person.  This  in  turn  infers  that: 

•  Cognitive  processes  are  linked  to  behaviour 

•  The  team  has  limited  attentional  resources 

•  Sometimes  important  knowledge  in  the  team  mind  is  not  retrieved 

•  The  team  mind  depends  on  meta  cognition 

•  The  team  mind  is  affected  by  motivation 

•  A  team  must  learn  to  co-ordinate  its  actions 

•  Teams  can  draw  inferences  (by  recognising  patterns  in  information  held  by  various 
team  members) 

This  model  infers  that  literature  related  to  individual  decision-making  will  also  be 
applicable  to  understanding  team  decision-making.  However,  within  this  idea  it  is  still 
unclear  as  to  how  the  "team  mind"  works. 

6.5  Summarising  Teamwork  and  Naturalistic  Decision  Making 

Because  teams  are  often  integral  to  NDM  performance  it  is  important  to  understand 
decision-making  in  the  context  of  a  team.  Despite  the  lack  of  consensus  in  the  literature  on 
defining  groups  and  teams,  it  is  believed  that  a  team  is  a  type  of  group  in  which  each 
member  has  a  role  and  works  interdependently  toward  a  common  meaningful  goal.  This 
definition  is  appropriate  to  military  HQ  teams.  Social  psychological  research  has  identified 
both  social  and  structural  characteristics  common  to  groups  of  people  working  together. 
These  include:  cohesiveness,  socialisation,  team  norms,  roles  and  communication 
networks.  Many  theories  have  been  developed  to  describe  team  functioning.  These  fall 
into  eight  main  categories  and  are  summarised  by  Paris,  Salas  and  Cannon-Bowers  (2000). 
Factors  internal  and  external  to  the  individuals  have  been  identified  as  being  predictive  of 
effective  teamwork.  Internal  factors  have  been  identified  as:  cognitions,  skills  and 
attitudes.  External  factors  are:  contextual  factors,  structural  factors,  team  design  factors, 
process  factors,  and  contingency  factors.  Recent  research  has  focused  on  the  importance  of 
SMM  in  effective  teams.  SMM  guide  how  team  members  interact,  and  also  facilitate 
decision-making  in  teams.  Finally,  Klein  and  Thorsden  (1990)  have  suggested  that  for 
research  and  modelling  purposes  a  "team  mind"  can  be  established  as  analogous  to  the 
mind  of  one  person.  This  offers  a  range  of  implications  for  future  research  and  design. 
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7.  Decision  Error 


Decision  error  is  one  of  the  eventual  observable  NDM  outcomes  that  research  attempts  to 
reduce.  Ultimately,  if  NDM  always  produced  optimal  outcomes,  there  would  be  less  need 
to  research  its  intricacies.  However,  as  illustrated  at  the  beginning  of  this  report, 
catastrophic  errors  can  and  do  occur. 

There  are  inherent  problems  in  defining  what  constitutes  a  good  decision.  It  has  been 
suggested  that  a  good  decision  can  be  defined  as  one  that  maximises  expected  utility 
(Edwards,  Kiss,  Majone,  &  Toda,  1984).  However,  this  definition  is  open  to  interpretation. 
In  contrast  to  this,  one  could  focus  on  defining  what  constitutes  a  decision  error.  Reason 
(1990)  defined  decision  error  as  any  decision  that  fails  to  achieve  the  outcome  originally 
intended.  This  definition  allows  for  decisions  to  either  be  correct  or  incorrect.  However,  in 
reality  it  seems  that  even  if  a  decision  matches  up  with  expectations  there  are  still  varying 
degrees  of  success  ("decision  quality").  However  measuring  "decision  quality"  seems 
problematic.  It  is  particularly  difficult  to  measure  the  decision  quality  of  military 
commanders  because  there  are  so  few  observable  outcomes  that  are  not  also  affected  by 
factors  other  than  the  decision  made.  Currently,  researchers  are  still  attempting  to  develop 
valid  methods  with  which  to  explore  decision  quality.  To  date  many  methods  have 
assumed  that  what  experts'  do  is  "proficient"  decision-making.  This  assumption  may  not 
be  entirely  appropriate  however,  because,  even  experts  have  been  reported  to  make 
mistakes  (Dixon,  1976;  Orasanu,  Dimukes,  &  Fischer,  1993;  Orasanu,  Martin  &  Davidson, 
2001).  Therefore,  in  a  study  of  expertise  it  is  important  to  recognise  this  factor  and  attempt 
to  establish  an  understanding  of  how  and  why  these  decision  errors  occur.  It  can  be 
argued  that  if  we  can  develop  a  good  understanding  of  what  causes  decision  errors,  and 
implement  appropriate  aids  and  interventions,  we  should  be  able  to  significantly  decrease 
decision-making  error. 

7.1  Explanations  for  Decision  Error 

Klein  (1998)  broadly  defines  three  main  categories  of  decision  error. 

•  Lack  of  experience  of  the  decision-maker. 

•  Lack  of  information  (this  impacts  on  developing  a  good  situation  awareness). 

•  Poor  mental  simulation  (where  decision-makers  would  notice  the  signs  of  a  problem, 
but  explain  them  away.) 

Two  of  these  factors  are  associated  with  the  cognition  of  the  decision-maker  (lack  of 
experience,  poor  mental  simulation)  and  the  third  is  an  external  factor  (lack  of 
information). 

Consistent  with  Klein's  (1998)  analysis,  there  have  been  two  schools  of  thought  explaining 
decision  error.  The  first  one  locates  the  error  as  occurring  within  the  cognitive  processing 
of  the  decision-maker.  The  second  locates  the  cause  of  error  external  to  the  decision-maker. 
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and  looks  at  the  context  surrounding  the  decision.  It  is  likely,  however,  that  there  is  an 
interrelationship  between  context  and  cognition  that  contributes  to  NDM  error. 

7.1.1  Context 

The  context  surrounding  naturalistic  decision-making  is  seen  as  increasingly  important 
(Lewandowsky  &  Kirsner,  2000;  Orasanu,  Martin  and  Davidson,  2001;  Schliemann  & 
Carraher,  1993).  Woods,  Johannesen,  Cook  and  Sarter  (1993)  suggest  that  instead  of 
focusing  on  an  individual's  internal  cognitions,  if  attention  is  given  to  the  available 
information,  the  goals  pursued  and  the  level  of  experience,  it  becomes  easier  to  remove 
blame  from  the  decision-maker  and  investigate  the  poor  situation. 

Orasanu,  Martin  and  Davidson  (2001)  investigated  cognitive  and  contextual  factors  in 
aviation  error.  They  suggested  that  experts  could  make  errors  in  their  decision-making  in 
one  of  two  processes: 

•  Developing  situation  awareness 

•  Selecting  an  appropriate  action 

They  found  from  analysing  aviation  accident  reports  that  four  contextual  factors  seemed 
to  be  influencing  expert  error.  They  were: 

•  Ambiguity 

•  Dynamic  Risk 

•  Organisational  and  Social  Pressures 

•  Stress  (working  memory  capacity  [Hockey,  1979]) 

These  four  areas  of  error  fit  into  different  types  of  contextual  frames. 

•  Relational  Context  -  This  refers  to  the  relational  networks,  and  collaborative  processes 
between  individuals.  It  also  takes  into  account  the  mood  and  morale  created  by  these 
interactions.  Importantly,  it  includes  analysis  of  communication  (both  implicit  and  explicit), 
and  the  resulting  shared  understanding.  Therefore,  three  sub-components  could  be: 

o  Relational  networks 
o  Communication  (implicit/ explicit) 
o  Morale/  Mood 

This  is  where  "organisational  and  social  pressures"  as  a  predictive  category  for  error 
would  fit  in  (Orasanu  et  al.,  2001). 

•  Physical  Context  -  This  refers  to  the  physical  environment  in  which  decision-makers  find 
themselves.  This  environment  can  have  elements  that  are  dynamic,  and  others  that  are 
constant.  For  example,  for  an  emergency  doctor,  the  space  s/he  occupies  remains  relatively 
constant.  However,  the  equipment  being  used  by  the  staff  with  whom  s/he  is  working  and  the 
patients  s/he  is  treating  are  dynamic.  The  dynamic  environment  often  contains  cues  that 
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trigger  expert  schema  and  decision-making.  This  concept  of  physical  context  then  could  be 
divided  into  two  sub-components: 
o  Stable  environment  (relatively  unchanging) 
o  Dynamic  environment  (constantly  changing) 

•  Specific  cues  (triggers  for  expert  schema) 

This  dynamic  environment  would  create  ambiguous  information  at  times  feeding  into 
decision-making.  Thus,  as  mentioned  by  Orasanu  et  al.  (2001),  ambiguity,  resulting  from  a 
dynamic  environment  is  predictive  of  error. 

•  Emotional  Context  -  This  refers  to  the  human  emotions  entwined  in  the  scenario  that  could 
affect  perception,  and/or  performance.  This  could  refer  to: 

o  Stress  and  anxiety  levels 

o  Personality  factors  (associated  with  individuals) 
o  Emotion  triggering  events  (such  as  death) 

Stress,  as  described  by  Orasanu  et  al.  (2001),  would  fit  in  this  area. 

•  Temporal  Context  -  This  refers  to  the  timeframes  in  which  the  event  is  situated.  The  length 
of  time  taken  for  a  decision  to  be  made  or  the  available  time  for  that  decision  to  be  made  would 
fall  into  this  category.  Sub-components  would  be: 

o  Time  of  day 
o  Length  of  event 

Risk  changing  over  time  (dynamic  risk;  Orasanu,  Martin  &  Davidson,  2001)  would  be  an 
artifact  of  the  temporal  and  physical  contexts  interrelating. 

7.1.2  Where  Context  Meets  Cognition 

Hutchins  (1996)  believes  that,  "boundaries  between  individuals  and  context  should  be 
softened. . .  The  apparent  necessity  for  drawing  such  a  boundary  is  in  part  a  side  effect  of 
the  attempt  to  deal  with  the  individual  as  an  isolated  unit  of  cognitive  analysis,  without 
first  locating  the  individual  in  a  culturally  constructed  world."  (p.  287-288).  It  may  be  that 
in  order  to  understand  NDM  error  more  fully,  both  context  and  cognition  need  to  be 
examined. 

In  classical  decision  research,  a  number  of  decision  biases  have  been  identified  that  cannot 
be  explained  by  the  theory,  and  that  produce  outcomes  that  would  deviate  from 
expectations.  For  example,  Kahneman,  Slovic  and  Tversky  (1982)  show  that  decision¬ 
makers  usually  use  "cognitive  shortcuts"  that  produce  answers  with  less  cognitive  effort, 
but  there  is  greater  risk  of  error.  These  are  referred  to  as  heuristics.  These  same  biases  may 
relate  to  NDM.  Some  examples  that  may  be  relevant  to  the  military  climate  are: 

•  Cognitive  dissonance  theory  (Festinger,  1957) 

•  Memory  and  hindsight  biases 

•  Context  dependence 
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Festinger  (1957)  proposed  the  theory  of  cognitive  dissonance,  based  on  observing  typical 
human  behaviour  of  selective  perception.  People  experience  cognitive  dissonance  when 
they  hold  two  thoughts  that  are  contradictory  or  incompatible.  He  proposed  that, 
whenever  possible  people  attempt  to  reduce  dissonance  (an  aversive  condition).  In 
essence,  this  explains  why  people  often  change  their  behaviour  to  match  their  beliefs,  or 
vice  versa.  In  a  military  environment  an  officer  may  justify  a  decision  s/he  has  made  and 
ignore  the  evidence  of  risks  (Dixon,  1976).  There  are  instances  of  both  pre-decisional  and 
post-decisional  effects  of  cognitive  dissonance. 

•  An  example  of  pre-decisional  dissonance  was  documented  by  Kantola,  Syme  and 
Campbell  (1984).  They  found  that  when  people  who  were  reported  as  using  large 
amounts  of  electricity  were  confronted  with  the  reminder  of  an  earlier  conservation 
endorsement  they  had  made,  they  tended  to  cut  their  consumption  significantly. 

•  An  example  of  post  decisional  dissonance  is  presented  by  Frenkel  and  Doob  (1976). 
After  examining  a  federal  election  they  found  that  when  comparing  respondents 
attitudes  before  and  after  the  voting  process,  voters  were  more  likely  to  believe  that 
their  candidate  was  the  best  one  and  had  the  best  chance  to  win  after  they  had  voted. 

Further  biases  that  are  relevant  to  the  military  context  are  memory  and  hindsight  biases.  A 
critical  fact  that  cannot  be  ignored  when  thinking  about  the  role  of  previous  experience  in 
formulating  mental  models  is  that  memory  is  not  always  an  accurate  storage  of  past 
information.  As  we  recall  a  memory,  we  also  need  to  reconstruct  it  mentally  (Loftus,  1980; 
Myers,  1990).  Hindsight  bias  is  the  "tendency  to  view  what  has  already  happened  as 
relatively  inevitable  and  obvious"  (Pious,  1993,  p.  35).  Therefore,  the  idea  of  drawing  upon 
memory  to  feed  previous  experiences  into  current  decision-making  is  fraught  with 
problems.  It  is  possible  that  in  the  process  of  recalling  previous  experience  Commanders 
could  overestimate  how  obvious  a  particular  course  of  action  was  in  the  past.  Hence,  this 
could  affect  future  decisions. 

Psychological  research  has  also  demonstrated  that  context  in  a  most  basic  sense  influences 
judgement.  Three  such  biases  due  to  information  presentation  are: 

•  The  Primacy  Effect 

Characteristics  presented  early  in  information  gathering  influence  impressions  more 
than  those  gathered  later  (Asch,  1946). 

•  The  Recency  Effect 

When  people  are  able  to  remember  the  most  recent  piece  of  information  more  clearly 
than  the  initial  one,  it  will  tend  to  influence  their  impressions  most  (Miller  & 
Campbell,  1959).  Recency  effects  are  a  function  of  recall.  In  a  military  domain, 
primacy  effects  may  be  more  common  than  recency  effects  because  decisions  are 
immediately  linked  to  information  gathering  (Hoch,  1984) 
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•  Halo  Effects 

The  halo  effect  illustrates  that  people's  reactions  to  stimuli  are  context  dependent 
(Pious,  1993).  It  has  been  found  that  when  people  rate  others  on  various  attributes  the 
ratings  are  often  correlated  (Thorndike,  1920;  Cooper,  1981;  Feldman,  1986;  Landy  & 
Si  gall,  1974).  Therefore,  impressions  formed  about  a  stimulus  can  be  based  on  beliefs 
associated  with  a  particular  attribute.  For  example,  if  a  certain  member  of  the  HQ  staff 
was  thought  of  as  radical,  less  attention  may  be  placed  on  his  input  to  the  planning/ 
decision-making. 

Other  decision  heuristics  that  may  be  applicable  to  decision-making  in  the  military  are: 
o  Representativeness 
o  The  Law  of  Small  Numbers 
o  The  Hot  Hand 
o  Neglecting  base  rates 
o  Nonregressive  prediction 
o  Clinical  vs  Actual  Prediction 
o  Availability 
o  Confirmation 

A  more  detailed  discussion  of  these  can  be  found  in  Pious  (1993).  It  is  important  in 
establishing  an  understanding  of  expertise  in  NDM  to  be  aware  of  these  heuristics  and 
biases  that  would  potentially  affect  decision-making  in  a  military  context. 

Recent  research  in  an  NDM  setting  has  supported  the  idea  that  cognition  and  context  are 
important  to  consider  together  to  begin  to  understand  decision  error  (Lewandowsky  & 
Kirsner,  2000;  Johnson-Laird,  1999). 

7.1.3  Recent  Research  on  Context  and  Cognition  in  an  NDM  Environment 

Two  ideas  that  have  recently  been  suggested  to  explain  certain  decision  errors  in  an  NDM 
environment  are: 

•  Knowledge  partitioning 

•  Falsity 

These  will  be  discussed  in  turn. 

It  has  been  argued  in  the  literature  that  expertise  is  domain  specific  and  inflexible.  One 
example  of  this  is  provided  by  Frensch  and  Sternberg  (1989)  who  studied  expert  and 
novice  bridge  players.  They  found  that  when  the  rules  of  bridge  were  changed,  there  was 
no  significant  difference  in  performance  between  expert  and  novice  groups.  The  "expert 
skills"  of  the  bridge  players  were  of  no  advantage  when  the  rules  were  changed.  This 
illustrates  the  importance  of  understanding  the  limits  of  the  domain  in  which  one  holds 
expertise. 
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The  integrated  nature  of  expertise  within  a  defined  domain  is  echoed  throughout  the  field 
(Glaser,  1996).  Berdard  and  Chi  (1992)  write  that  "experts  have  more  stronger  links  among 
concepts,  suggesting  there  is  a  greater  degree  of  connectedness  and  cross  referencing,  and 
the  pattern  of  connections  and  cross  referencing  can  result  in  a  better  structure"  (p.  136). 
However,  the  'within-domain'  integrated  nature  of  expertise  is  questioned  by 
Lew  and  ow  sky  and  Kirsner  (2000).  They  found  that  when  performance  was  compared 
across  different  contexts,  but  in  the  same  domain  (identical  physical  conditions),  the 
predictions  that  were  made  were  completely  opposing.  They  go  on  to  suggest  that 
expertise  may  "comprise  separate,  and  sometimes  even  mutually  exclusive  components  of 
knowledge"  (p.  295). 

The  testing  tool  used  by  Lewandowsky  and  Kirsner  (2000)  was  a  bushfire  computer 
simulation  prediction  task.  The  experiments  manipulated  context  within  this  tool  (experts 
familiar  with  situations).  The  metrics  taken  were:  how  accurate  expert  predictions  were, 
physical  model  compared  with  expert's  model,  and  how  do  experts  cognitively  evaluated 
combined  effects  of  determinant  variables  (eg.  wind  and  slope).  The  outcome  was  that 
experts  were  identified  from  novices  not  by  their  use  of  strategy  (for  example  wind 
dominance  over  slope)  but  their  context  specificity  of  performance. 

In  response  to  these  findings,  Lewandowsky  and  Kirsner  (2000)  have  proposed  that 
experts  store  information  pertaining  to  separate  areas  of  their  experience  in  different 
"packages".  It  is  suggested  that  each  package  is  integrated  internally,  but  integration 
between  packages  proves  difficult.  This  independence  across  packages  could  cause  the 
inconsistencies  and  mistakes  that  have  been  observed  in  expert  performance.  The  access  to 
these  knowledge  packages  is  guided  by  the  problem  context.  From  this,  it  is  speculated 
that  partitioning  could  be  avoided  if  training  instructions  or  diagrams  foster  integration  of 
predictors  (eg.  wind  speed  and  slope  for  a  fire  fighting  context). 

This  finding  is  comparable  with  those  found  by  Schliemann  and  Carraher  (1993).  In  this 
study,  two  mathematically  identical  problems  were  given  to  expert  cooks.  They  were: 

•  If  2  kg  of  rice  cost  5  cruzeiros,  how  much  do  you  have  to  pay  for  3  kg? 

•  To  make  a  cake  with  2  cups  of  flour  you  need  5  spoonfuls  of  water.  How  many 
spoonfuls  do  you  need  for  3  cups  of  flour? 

Both  problems  are  relevant  to  their  domain  of  expertise.  Interestingly  though,  in  the 
pricing  context  accuracy  of  response  equated  to  90%.  In  regards  to  the  recipe,  accuracy 
was  dramatically  lower  at  20%.  Schliemann  and  Carraher  (1993)  explained  this  in  terms  of 
the  kinds  of  cognitive  methods  the  cooks  might  generally  employ  in  such  situations.  In 
the  pricing  context,  calculation  of  an  accurate  answer  would  be  required  by  social 
convention.  However  in  regard  to  the  recipe  measurements  expert  cooks  might  generally 
use  estimations  in  order  to  generate  solutions.  This  result  also  emphasises  the  use  of 
different  ways  to  solve  a  problem  in  different  contexts  within  the  same  domain.  It  is 
reasoned  that  in  both  cases  described  expert  errors  could  be  explained  by  applying  a 
compartmental  view  of  expert  information  storage  and  retrieval. 
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The  second  example  of  a  recent  explanation  for  decision  error  is  falsity.  Johnson-Laird 
(1999)  has  pointed  out  the  cognitive  differences  when  dealing  with  either  true  or  false 
situations.  Johnson-Laird  (1999)  draws  on  the  concept  of  mental  models.  When  people 
reason  they  draw  on  their  mental  models,  which  act  like  a  truth  table.  Because  these  are 
held  in  working  memory,  the  adequacy  of  these  is  limited  by  the  capacity  of  working 
memory.  This  suffices  for  straightforward  problems,  but  when  one  has  to  deal  with 
complex,  false  information,  humans  tend  to  not  give  answers  based  on  a  complete  set  of 
logic.  When  we  are  told  a  piece  of  information  is  false  it  becomes  difficult  to  generate  all 
the  other  possibilities  for  what  the  true  piece  of  information  might  be  and  hold  them  all  in 
working  memory.  Experimental  research  has  been  carried  out  to  demonstrate  this  theory. 
For  example,  the  problem  below  was  presented  to  a  group  of  Princeton  University 
students.  They  were  told  that  only  one  of  the  following  statements  about  a  hand  of  cards 
was  true: 

•  There  is  a  king  in  the  hand,  or  an  ace,  or  both. 

•  There  is  a  queen  in  the  hand  or  an  ace,  or  both. 

•  There  is  a  Jack  in  the  hand,  or  a  ten  or  both. 

•  Is  it  possible  there  is  an  ace  in  the  hand? 

From  this  99%  of  the  students  answered  incorrectly.  Counter  intuitively,  the  answer  is 
"no".  At  first  glance  it  may  appear  that  there  can  be  an  ace  because  there  is  an  ace  in  the 
first  two  statements.  Flowever,  if  there  was  an  ace  then  it  would  follow  that  the  first  two 
statements  would  be  true  and  it  was  explicitly  stated  in  the  problem  formulation  that  only 
one  of  the  statements  could  be  true.  Therefore,  neither  of  the  first  two  options  can  be 
exclusively  true.  Johnson-Laird  (1999)  explains  that  when  we  are  told  a  piece  of 
information  is  false  it  becomes  difficult  to  generate  all  the  other  possibilities  for  what  the 
true  piece  of  information  might  be  and  hold  them  all  in  working  memory.  Therefore, 
Johnson-Laird  (1999)  suggests  humans  simply  ignore  the  falsity  in  order  to  process  the 
information  more  easily  (for  a  summary  of  falsity  see  Butavicius  &  Galanis,  in  press). 

7.1.4  Teamwork  and  Decision-Making  Error 

In  many  NDM  situations  a  team  of  individuals  are  required  to  assist  in  decision-making. 
Thus,  a  team  level  understanding  of  NDM  error  is  also  required. 

It  has  been  suggested  that  group  decision-making  is  often  inferior  to  individual  decision¬ 
making  (Brown,  1988).  Postmes  and  Lea  (2000)  suggest  that  this  can  be  attributed  to: 

•  Process  dysfunctions 

•  Social  dysfunctions 

Process  dysfunctions  arise  because  of  structural  characteristics  in  the  group  setting  that 
may  affect  decision-making.  Examples  could  be  disproportionate  chances  to  communicate 
or  participate. 
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Social  dysfunctions  are  caused  by  "limitations  inherent  in  the  structure  and  form  of 
meetings"  (Postmes  &  Lea,  2000,  p.  1253).  Examples  of  these  processes  are:  evaluation 
apprehension,  socialising,  conformity  pressures,  social  loafing,  domination  due  to  status 
imbalance,  free  riding,  group  think  (Steiner,  1972;  Stroebe  &  Diehl,  1994;  See  Appendix  B 
for  definitions). 

Jones  and  Roelofsma  (2000)  add  a  third  causal  factor  to  this  suggestion.  They  suggest  that 
in  terms  of  decision-making  error  in  teams,  there  are  cognitive  triggers,  as  well  as 
organizational  and  social  causes. 

Therefore,  any  decision  made  in  a  military  HQ  environment  should  be  considered  at  three 
levels:  cognitive  (individual),  organisational  (team/ context)  and  social  (team/ context).  As 
individual  cognitive  biases  have  already  been  described,  the  focus  will  now  be  placed  on 
organisational  and  social  triggers  for  decision  error. 

7.1.4.1  Organisational  Triggers  for  Decision  Error 

It  is  interesting  to  note  that  many  decision  errors  that  have  been  subsequently  analysed 
have  been  attributed  to  high  level  management  decisions.  For  example,  Bruggink  (1985) 
found  that  65%  of  aircraft  accidents  analysed  had  been  influenced  by  policy  factors.  This 
makes  it  obvious  that  decision-making  at  a  certain  level  in  the  military  must  be  looked  at 
in  the  organizational  context  in  which  it  occurs. 

From  the  field  of  organisational  psychology  phenomena  such  as:  organisational  culture, 
group  structure,  power  structure,  role  structure  and  communication  structure  have  been 
identified  as  organisational  factors  that  may  potentially  influence  performance  outcomes 
(For  a  summary  see  Buchanan  &  Huczynski,  1997). 

One  area  of  research  which  examines  organisational  dynamics  traditionally  associated 
with  military  performance  is  'Unit  Climate  Profiling'  (UCP;  Murphy  &  Farley,  2000).  'Unit 
climate'  is  similar  to  the  construct  of  'organisational  climate'  that  is  widely  used  in  the 
organisational  psychology  literature  (e.g.,  Argyle,  1972;  Carey,  1998;  Schneider,  1990).  Unit 
climate  is  made  up  of  several  dimensions  presumed  to  be  antecedents  of  individual,  team 
and  unit  readiness  to  undertake  military  duties. 

The  UCP  developed  from  a  conceptual  framework  (Wild,  1988)  that  attempted  to  predict 
combat  effectiveness  from  postulated  readiness  indictors.  The  human  components  of 
combat  readiness  included:  aptitudes,  combat  proficiencies,  understanding  of  the  task(s), 
the  motivational  environment,  performance  motivation,  command  intent  and  leadership. 
Using  the  motivational  components  of  this  model.  Reeves  and  Hansen  (1989)  developed 
and  administered  a  psychometric  instrument,  the  Human  Dimension  of  Combat  Readiness 
Index  -  Experimental  (HDCRI-X),  to  a  sample  of  Canadian  soldiers.  Factor  analysis 
confirmed  the  existence  of  eight  independent  factors,  made  up  of  56  items,  which 
accounted  for  53.9%  of  total  variance.  The  eight  dimensions  were  labelled:  morale/ 
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cohesion,  leadership  skills,  ideology10,  professional  morale11,  and  confidence  in  leadership 
at  four  distinct  rank  levels.  It  is  noted  that  the  sample  size  used  for  this  analysis  was  less 
than  satisfactory  (a  general  recommendation  is  10  subjects  for  each  item  present  in  a  scale 
(Tabachnick  &  Fidell,  2001). 

The  resultant  56-item  HDCRI-X  was  used  by  Farley  (1995)  with  Canadian  soldiers 
deployed  to  Bosnia.  During  this  research,  he  pragmatically  refined  the  design,  analysis 
and  use  of  the  measure  and  renamed  it  the  'Unit  Climate  Profile'.  The  UCP  is  essentially  a 
tool  for  commanders  to  add  to  their  decision-making  'toolkit'. 

The  UCP  appears  to  have  been  useful  as  a  form  of  input  into  command  decision-making  in 
recent  peace  support  operations  (Farley,  2002).  However  formal  research  needs  to  be  done 
to  determine  whether  knowledge  of  unit  climate  factors  actually  affects  and  improves 
decision-making.  Interestingly,  previous  research  has  revealed  that  command  perceptions 
of  climate  factors  such  as  morale  and  confidence  in  leadership  can  be  at  considerable  odds 
with  the  perceptions  of  subordinates  (Korpi,  1965;  Eyres,  1998).  Korpi  (1965)  found  that 
Swedish  officers  had  "very  unreliable  notions  of  the  opinions  in  their  units"  (p.  302)  and 
that  officers  were  generally  unaware  that  their  perceptions  of  subordinate  morale  might  be 
inaccurate.  Indeed,  the  greater  confidence  officers  expressed  in  their  assessments  of  issues 
such  as  morale,  the  larger  the  error  in  their  predictions.  More  recently.  Eyres  (1998)  found 
that  although  officers  expressed  general  satisfaction  with  the  leadership  they  provided, 
subordinates  reported  they  were,  at  best,  indifferent  to  officer  leadership. 

These  findings  must  be  considered  in  the  context  of  decision-making.  If  commanders  do 
not  accurately  perceive  or  assess  prevailing  attitudes,  norms,  and  values  in  a  unit's  culture 
and  subcultures,  or  their  own  impact  on  subordinate  behaviour,  then  the  likely 
consequences  of  their  decisions  may  be  poorly  appreciated.  Research  on  the  influence  of 
poor  or  conflicting  perceptions  of  unit  climate  factors  on  decision  performance  appears 
warranted. 

7.1. 4.2  Social  Triggers  for  Decision  Error 

Explanations  of  social  influences  on  decision-making  draw  upon  a  large  field  of  research, 
particularly  stemming  from  the  field  of  Social  Psychology.  Jones  and  Roelofsma  (2000) 
suggest  that  some  important  social/ group  biases  to  consider  in  team  decision-making  are: 
false  consensus  effect,  groupthink,  group  polarisation,  and  group  escalation  of 
commitment. 

The  false  consensus  effect  is  the  tendency  that  people  have  to  overestimate  the  degree  to 
which  their  own  choices  and  judgements  are  similar  to  others  (Ross,  Greene  &  House, 
1977).  This  bias  can  affect  teams  in  terms  of  assumptions  that  may  be  made  when 


10  Ideology  refers  to  internalised  normative  beliefs  associated  with  military  culture  such  as  the 
legitimacy  of  the  use  of  force  and  commitment  to  military  ethos 

11  Professional  morale  refers  to  soldiers'  confidence  in  their  military  skills  and  abilities 
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incomplete  information  is  received  upon  which  to  base  decisions.  Assumptions  may  be 
made  regarding:  members  of  one's  own  team,  the  enemy,  other  teams  or  individuals 
within  the  organisation.  Fischhoff  and  Johnson  (1997)  have  put  forward  the  notion  that 
false  consensus  may  result  from  an  inaccurate  mental  model  of  the  group  about  which  one 
is  making  an  assumption.  This  bias  requires  further  research  within  a  command  and 
control  setting  in  order  to  more  fully  understand  how  it  may  factor  into  decision  error. 

Groupthink  is  a  mode  of  thinking  where  the  group's  desire  to  agree  overrides  their  desire 
to  come  up  with  the  best  decision  (Janis,  1972).  Groupthink  can  lead  to:  incomplete  survey 
of  alternatives,  consideration  of  too  few  objectives,  and  poor  information  search  (Janis  & 
Mann,  1977).  Despite  criticisms  of  the  theory  (Leana,  1985;  McCauley,  1989)  it  seems  to 
have  been  accepted  universally  as  a  phenomenon  worth  pursing  further  in  research 
(Adlag  &  Riggs  Fuller,  1993).  Thus  far,  groupthink  in  a  team  context  seems  not  to  have 
been  tested.  It  is  therefore  important  to  take  this  into  consideration  as  a  potential  team 
bias. 

Polarisation  is  the  tendency  for  the  group  members  to  make  decisions  that  are  more 
extreme  than  the  mean  tendency,  but  in  the  same  direction  as  that  tendency  (Lamm,  1988). 
The  two  types  of  polarisation  are  risky  shift  (the  tendency  to  choose  a  course  of  action 
more  risky  than  the  group  norm)  and  cautious  shift  (the  tendency  to  choose  a  course  of 
action  more  cautious  than  the  group  norm).  One  explanation  for  this  phenomenon  takes 
into  account  group  norms.  It  is  understood  that  more  extreme  decisions  are  made  in  order 
to  confirm  group  norms.  Little  research  seems  to  have  been  done  in  a  command  and 
control  setting  on  this  phenomenon,  particularly  in  terms  of  the  effects  of  time  pressure, 
high  stakes,  and  uncertainty. 

Group  escalation  of  commitment  is  the  tendency  of  groups  to  pursue  a  particular  course  of 
action  even  in  light  of  evidence  that  it  is  failing.  Obviously,  within  this  bias,  the  issue  of 
leadership  arises.  A  particular  type  of  group  escalation  occurs  in  competitive  situations. 
This  can  be  illustrated  by  considering  a  price  war  between  two  companies.  Each  company 
continues  to  lower  the  price  of  their  product  until  they  begin  reaping  losses  from  the 
discounts  applied  to  their  products.  Rationally,  this  should  end  the  price  war,  but  as  was 
the  case  with  a  price  war  between  American  Airlines  and  Northwest  Airlines  in  1992,  the 
war  continues  past  this  point  (Jones  &  Roelofsma,  2000).  Clearly,  this  presents  an  area  to 
be  considered  in  future  command  and  control  team  research.  Possible  benefits  may  be 
produced  if  one  could  convince  the  enemy  to  escalate  his  commitment. 

As  well  as  social  biases  that  may  affect  decision  outcome,  there  are  several  other 
documented  group  processes  that  are  deemed  likely  to  impact  on  team  decision  quality. 
An  example  of  this  is  the  process  known  as  "brainstorming".  Generating  novel  ideas  is 
important  in  a  military  context.  Often  this  is  achieved  through  a  process  similar  to 
"brainstorming"  (Osborn,  1957).  Despite  evidence  that  groups  left  to  brainstorm  actually 
do  generate  more  ideas  than  the  individual  members,  it  has  not  been  found  that  they  are 
more  creative  (Diehl  &  Stroebe,  1987;  Mullen,  Johnson  &  Salas,  1991).  In  fact,  it  has  been 
established  instead  that  groups  where  individuals  create  their  own  ideas  without 
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interacting,  are  more  creative  than  those  that  work  as  a  group  (Paulus,  Dzindolet,  Poletes 
&  Camacho,  1993).  The  reasons  that  this  might  be  the  case  include: 

•  Evaluation  apprehension  -  Group  members  concerned  with  making  a  good 
impression  introduces  a  degree  of  self-censorship. 

•  Social  loafing  -  Motivation  is  lost  because  the  task  is  collective  in  nature. 

•  Production  blocking  -  Because  individuals  have  to  contend  with  others  generating 
ideas  at  the  same  time  they  are  often  stifled  creatively. 

Numerous  conditions  for  decision  error  exist  both  at  an  individual  cognitive  level,  and 
also  at  a  team  level  (social  and  organizational).  These  need  to  be  considered  in  research 
dealing  with  the  Commander's  decision-making  within  the  context  of  the  team. 

7.2  Summarising  Decision  Error 

An  important  aim  for  NDM  research  is  to  reduce  the  occurrence  of  decision  error. 
Therefore  in  order  to  facilitate  this  an  understanding  of  possible  triggers  for  decision  error 
is  needed.  There  have  been  a  number  of  explanations  put  forward,  and  most  tend  to  focus 
on  context  and  its  effect  on  cognition.  Contextual  causes  of  decision  error  fall  into  four 
categories:  relational  context,  physical  context,  emotional  context  and  temporal  context. 
Interestingly,  when  the  effects  of  context  on  cognition  are  examined  various  types  of 
decision  errors  have  been  identified.  Typically  humans  try  to  simplify  cognitive  processes. 
They  use  heuristics  (otherwise  known  as  "rules  of  thumb")  to  do  this.  Decision  heuristics 
can  lead  to  quicker  decisions,  but  they  tend  to  present  more  risk  of  error.  Several  heuristics 
that  have  been  identified  in  the  literature  can  be  applied  to  a  military  setting.  Examples 
are:  cognitive  dissonance,  recency  and  primacy  biases,  and  halo  effects.  Biases  that  have 
recently  received  attention  in  the  literature  are  knowledge  partitioning  and  falsity.  Biases 
do  not  only  operate  at  an  individual  level  but  also  in  a  team  setting.  These  occur  as  process 
dysfunctions  and  social  dysfunctions.  They  include  such  things  as:  false  consensus  effect, 
groupthink,  group  polarisation,  and  group  escalation  of  commitment. 


8.  Conclusions  on  Expertise  in  NDM 

8.1  Summary 

It  seems  apparent  that  the  topic  of  expertise  in  NDM  is  an  important  concept  for 
researchers  concerned  with  military  decision-making.  At  the  outset  of  this  report,  the  aims 
were  given  as:  to  present  the  relevant  background  information  required  to  begin  to 
understand  the  field  of  NDM,  how  expertise  is  understood  within  this  field,  and  how  an 
expert  functions  within  a  team  environment.  The  main  topics  discussed  have  been: 
decision  theories,  previous  experience  and  expert  functioning,  levels  of  expertise, 
teamwork  and  naturalistic  decision-making,  and  expert  error.  These  topics  are  essential  in 
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providing  a  platform  for  future  research  into  command  decision-making.  They  isolate 
concepts  that  must  be  understood  in  order  to  improve  the  development  of  command 
support  tools,  and  training  interventions  in  the  Australian  Army. 

Firstly,  this  review  looked  broadly  at  the  history  of  decision  research.  Traditional 
analytical  models  have  been  presented,  and  critiqued  as  explanations  for  decision-making 
in  naturalistic  environments  because  of  their  reliance  on  assumptions  of  probability. 
Research  has  found  that  humans  do  not  always  behave  in  this  manner.  This,  coupled  with 
the  understanding  that  classical  decision  theories  failed  to  explain  decision-making  in  an 
environment  such  as  the  military,  led  to  the  search  for  alternative  models. 

The  understanding  that  the  military  operates  in  an  environment  very  different  to  those  in 
which  much  of  the  traditional  decision-making  research  was  performed,  led  to  the 
establishment  of  the  NDM  theory.  This  in  turn  provided  a  platform  for  more  recent 
models  of  decision-making,  such  as  the  recognition  primed  decision-making  model  and 
the  schema  and  mental  models  in  RPD  theory  of  expertise.  These  theories  take  into 
account  the  short  time  frames,  uncertainty,  multiple  goals,  and  high  stakes  associated  with 
the  NDM  process.  From  these  theories,  important  ideas  that  have  been  introduced  are: 
situation  awareness,  the  use  of  previous  experience,  schemata  and  mental  models. 

Models  that  describe  NDM  emphasise  the  importance  of  experience  to  feed  into  decision¬ 
making.  Therefore,  understanding  how  experience  affects  decision-making  is  another 
important  area.  Much  of  the  research  on  expertise  predates  NDM  theory,  but  its  principles 
are  still  applicable  to  the  emerging  field.  Glaser  and  Chi  (1988)  carried  out  a  review  of 
research  on  expertise  and  outlined  a  number  of  its  facets  that  were  common  across 
domains.  Most  importantly,  it  seems  that  seems  that  experts  use  particular  cognitive 
strategies  in  order  to  complement  their  declarative,  perceptual  and  cognitive  knowledge. 
This  gains  them  advantage  over  their  novice  counterparts.  Thus,  investigation  into 
expertise  is  largely  concerned  with  the  encoding  and  storing  of  knowledge  based  on 
experience.  An  important  cognitive  model  of  how  this  experience  is  stored  in  memory  has 
been  proposed  by  Noble  (1993).  This  model  describes  how  previous  experience  feeds  into 
decision-making,  and  how  incomplete  information  is  handled.  This  information  is  directly 
applicable  to  the  models  previously  mentioned,  as  it  provides  a  deeper  understanding  of 
the  perceptions  and  processes  undertaken  in  NDM.  Thus  its  incorporation  into  a  more 
holistic  understanding  of  expert  decision-making  is  integral. 

Expertise  underlies  NDM,  and  is  based  on  gaining  a  repertoire  of  relevant  experiences  to 
feed  into  situation  assessment  and  decision-making.  Understanding  the  development  of 
expertise  looks  at  how  this  experience  is  gained,  and  describes  how  the  cognitive  skills 
associated  with  NDM  emerge.  Welford's  (1968)  model  of  the  storage  of  skill  in  memory 
has  been  discussed;  as  have  descriptive  models  of  development  of  expertise  proposed  by 
Anderson  (1991),  Dreyfus  (1972),  Fitts  and  Posner  (1967;  cited  in  Shanteau,  1992),  and 
Rasmussen  (1983).  These  models  are  all  similar  in  that  they  describe  decision  skills  as 
beginning  as  variable  and  awkward,  and  progressing  to  a  point  where  decisions  become 
intuitive,  and  responses  more  immediate  and  logical  to  the  decision-maker.  The 


44 


DSTO-GD-0429 


differences  lie  in  the  division  of  these  continuums  into  identifiable  stages.  Dreyfus's  (1972) 
model  proposes  that  five  levels  of  skill  competency  are  identifiable,  as  one  becomes  an 
expert.  However  distinctions  between  these  stages  seem  to  be  less  obvious  compared  with 
those  suggested  by  Rasmussen  (1983).  The  author  suggests  that  Rasmussen's  (1983)  model 
provides  a  good  illustration  of  expert  development  based  on  three  levels  of  skill 
competency.  As  NDM  expertise  is  developed,  the  four  types  of  skills  that  are  developed 
are:  performance,  judgement,  perception  and  locus  of  control  (Glaser,  1987).  Therefore  we 
can  begin  to  establish  a  cognitive  explanation  for  expert  decision-making,  and  how  this 
may  emerge  with  repetition  (experience).  Understanding  this  feeds  directly  into 
developing  training  methods. 

It  was  noted  that  for  application  to  the  real  world,  it  is  important  to  take  into  consideration 
the  fact  that  military  decision-makers  (especially  those  working  in  a  HQ  environment)  are 
part  of  a  team.  Therefore,  this  context  must  be  understood.  Social  psychological  research 
has  provided  several  characteristics,  both  social  and  structural,  common  to  groups  of 
people  working  together.  These  include:  cohesiveness,  socialisation,  team  norms,  roles  and 
communication  networks.  Many  theories  have  been  developed  to  describe  team 
functioning.  Also,  factors  internal  and  external  to  the  individuals  have  been  identified  as 
being  predictive  of  effective  teamwork.  Internal  factors  have  been  identified  as:  cognitions, 
skills  and  attitudes.  External  factors  are:  contextual  factors,  structural  factors,  team  design 
factors,  process  factors,  and  contingency  factors.  Research  on  mental  models  and  the 
"team  mind"  was  also  discussed.  These  concepts  are  important  because  they  provide 
theories  that  account  for  the  implicit  communication  through  NDM.  They  also  provide 
implications  for  future  research  and  design  with  teams. 

As  outlined  at  the  very  beginning  of  the  report,  errors  in  NDM  can  be  quite  catastrophic 
because  the  stakes  are  so  high.  Therefore,  an  important  aim  for  NDM  research  is  to  reduce 
the  occurrence  of  decision  error.  In  order  to  facilitate  this,  an  understanding  of  decision 
error  and  its  possible  triggers  is  needed.  Many  explanations  have  been  provided  for 
decision-making  error.  They  mainly  focus  on  the  effect  of  context  on  cognition.  Contextual 
causes  of  decision  error  have  been  identified  as  falling  into  the  following  areas:  relational 
context,  physical  context,  emotional  context  and  temporal  context,  and  psychological. 
Psychological  context  refers  to  the  use  of  decision  heuristics  in  order  to  make  quicker 
decisions.  Several  heuristics  can  be  applied  to  a  military  setting.  Examples  are  cognitive 
dissonance,  and  biases  due  to  the  order  of  information  presentation  (eg.  primacy  effect, 
recency  effect,  halo  effects).  Biases  that  have  recently  received  attention  in  the  literature  are 
knowledge  partitioning  and  falsity.  Understanding  team  biases  is  also  important.  These 
occur  in  the  form  of  process  dysfunctions  and  social  dysfunctions.  They  include  such 
things  as:  false  consensus  effect,  groupthink,  group  polarisation,  and  group  escalation  of 
commitment.  Taking  both  individual  and  team  biases  into  consideration  in  any  research 
on  military  command  decision-making  is  important. 
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8.2  Further  Research 

The  success  of  future  research  hinges  on  the  ability  of  researchers  to  provide  more  holistic 
understanding  of  the  command  process,  and  to  identify  where  theories  fall  short  of 
providing  explanations  for  observable  phenomena. 

From  reviewing  the  literature,  four  major  areas  have  been  identified  as  requiring  further 
research  in  order  to  create  more  adequate  models  of  command  decision-making.  Research 
in  these  areas  will  positively  influence  decision  support  tools  and  training  techniques. 
These  areas  are: 

•  Expertise:  The  question  of  how  an  expert's  cognitive  functioning  differs  from  that  of  a 
novice  has  created  much  interest  and  debate.  It  is  imperative  at  this  stage  to  integrate 
previous  research  and  apply  current  understandings  to  real-world  command  decision 
situations.  Successful  research  in  this  area  could  allow  people  to  be  trained  to  think 
like  experts,  possibly  without  the  large  time  commitment  generally  associated  with 
becoming  an  expert.  Research  needs  to  focus  on  the  processes  surrounding  a 
commander's  situation  awareness  and  decision-making  strategies.  This  will  assist  in 
the  design  of  appropriate  decision-making  aids  and  interfaces. 

•  Context:  The  influence  that  context  has  on  a  commander's  decision-making  processes 
is  currently  unclear.  In  order  to  develop  appropriate  decision  aids,  the  commander's 
decision-making  processes  must  be  looked  at  in  the  context  of  the  environment,  the 
team  and  the  organisation  in  which  s/he  works.  If  this  is  not  achieved,  development 
of  an  oversimplified  model  could  lead  to  production  of  inadequate  decision  aids. 

•  Team  Processes:  The  diffuse  nature  of  decision-making  within  a  military  HQ  team 
makes  it  difficult  to  only  focus  on  the  individual  when  researching  decision-making. 
Understanding  team  processes  in  an  environment  such  as  a  military  HQ  is  important. 
This  is  especially  important  in  a  military  setting  where  the  Commander  controls  the 
decision-making  and  the  staff  facilitate  other  steps  of  the  process.  Currently  the 
importance  of  the  HQ  staff  in  the  Commander's  decision-making  process  does  not 
seem  to  be  fully  understood.  This  needs  to  be  addressed.  This  process  should  also  be 
compared  to  other  command  and  control  settings,  in  order  to  understand  the 
relevance  of  literature  focused  on  other  NDM  domains  for  application  to  a  military 
setting. 

•  Decision  Error:  Another  essential  outcome  from  the  investigation  of  expertise  is  to 
generate  an  understanding  of  how  expert  error  fits  into  NDM  in  the  military. 
Specifically,  in  a  team  context,  it  is  crucial  to  identify  the  critical  areas  in  which 
decision  biases  may  impede  decision  outcomes.  This  will  also  be  imperative  for  the 
development  of  appropriate  decision  aids  to  facilitate  expert  decision-making. 
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8.3  Military  Specific  Research  Issues 

When  applying  research  such  as  this  to  a  military  specific  environment,  a  number  of 
factors  should  also  be  taken  into  consideration.  These  are: 

•  The  hierarchical  organisation 

•  Organisational  guidelines/  Codes  of  practice 

•  Regiments  (Corps  affiliation) 

•  Reserve  and  full-time  officers 

Because  the  military  operates  as  a  hierarchical  organization,  and  soldiers  can  be  promoted 
through  the  ranks  as  a  reward  for  good  performance,  it  is  generally  assumed  that  higher 
ranked  officers  will  have  more  experience,  and  will  perform  better  than  their  subordinates. 
The  organizational  structure  of  the  military  has  developed  over  the  years  to  support  the 
requirement  to  be  able  to  operate  in  extreme  circumstances  that  create  stress  and 
uncertainty.  The  structure  should  produce  an  environment  in  which  stress  and  workload 
is  shared,  and  a  young  soldier  can  carry  out  their  duties  without  allowing  their  emotions, 
morals,  and  values  to  hinder  their  performance  in  a  battle  situation.  Realistically,  this  may 
not  be  entirely  the  case. 

Secondly,  because  the  Army  operates  in  a  complex  environment  where  decisions  need  to 
be  made  quickly,  many  standard  operating  procedures  and  guidelines  that  soldiers  are 
expected  to  follow  have  been  developed.  These  are  expected  to  automate  many  of  the 
drills  and  procedures  that  are  frequently  carried  out  by  the  soldiers.  Consequently,  any 
discussion  about  decision-making  in  the  military  must  take  this  factor  into  account. 

Also,  within  the  Army  there  are  different  units  (regiments)  established  to  perform 
different  functions.  For  example,  there  is  a  regiment  dedicated  to  using  armoured  vehicles, 
an  infantry  regiment,  and  an  air  defence  regiment  (amongst  others).  A  regiment  would 
typically  consist  of  approximately  700  soldiers.  Because  of  the  different  functions  of  the 
regiments,  it  is  possible  that  different  business  processes  may  be  the  norm.  This  is 
obviously  an  important  issue  for  consideration. 

Lastly,  an  issue  related  to  performance  of  experts  in  naturalistic  environments  is  that  of 
time  commitment.  It  may  be  the  case  that  experts  that  commit  less  time  to  their  field  of 
expertise  perform  at  a  different  level  compared  to  those  who  commit  more  time  and 
gather  more  practical  experience.  This  is  similar  in  other  fields,  where  there  are  both  part- 
time  and  full-time  experts  who  function  at  various  intervals  (eg.  the  CFS  vs  the  fire 
brigade,  army  reservists  vs  full-time  army,  part-time  vs  full-time  doctors,  and  first  aid 
staff).  This  could  be  the  product  of  a  shift  from  full-time  employment  as  the  norm  to 
contract  work  and  part-time  positions.  It  is  also  a  product  of  the  demands  involved  in 
many  of  these  organizations.  In  particular,  additional  staff  or  decision-makers  are  often 
required  during  emergency  situations.  At  other  times,  such  a  large  group  is  not  necessary. 
In  the  Australian  Army,  there  are  a  lot  of  competing  demands  on  officers'  time,  and  not 
adequate  personnel  to  handle  the  work  load.  Therefore,  increasingly  the  reserve  officers 
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are  called  upon  to  complete  these  tasks.  Therefore,  despite  being  less  "experienced"  they 
need  to  be  equipped  to  handle  the  job  just  as  effectively  as  full-time  officers.  This  issue 
cannot  be  ignored  by  researchers  investigating  expertise  in  NDM. 

In  summary,  there  are  obvious  directions  for  future  research  aimed  at  understanding  more 
clearly  how  a  Commander  may  use  his  expertise  in  a  team  situation  in  order  to  make 
decisions.  If  we  do  not  carry  out  research  on  the  areas  outlined:  expertise,  context, 
teamwork,  and  decision  error,  there  may  be  dangerous  gaps  in  our  knowledge  of 
command  decision-making.  These  gaps  in  knowledge  will  then  translate  into  sub-standard 
design  of  decision  support  tools,  and  training  procedures.  To  allude  back  to  the  anecdote 
at  the  beginning  of  this  report,  let  us  feel  secure  in  the  understanding  that  with  further 
research  and  understanding  of  expertise  in  NDM  catastrophic  errors  such  as  that 
described  should  be  dramatically  reduced. 
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Appendix  A:  Factors  Affecting  Teamwork 


(Adapted  from  Morgeson,  Aiman-Smith  and  Campion,  1997). 


Factor(s) 

Description 

Examples 

Contextual 

Factors 

Variables  that  pertain  to  the 
environment  in  which  the  team 
activity  is  embedded 

Culture 

Climate 

Training/Education  systems 
Reward  systems 

Information  systems 

Structural 

Factors 

Variables  impinging  primarily 
from  sources  external  to  the 
team,  but  may  include  some 
internal  to  the  team  (eg.  Team 
organisation) 

Physical  environment 
Organisational  arrangements 
Technological  systems 

Team  Design 
Factors 

Variables  inherent  to  the  team 
itself  and  the  way  in  which  it 
functions 

Work  Design 

Task  interdependence 

Team  size/ composition 
Leadership 

Process 

Factors 

Variables  inherent  to  the  team 
itself  and  the  way  in  which  it 
functions 

Boundary  management 

Task  cohesion 

Performance  norms 
Communication 

Team  interactions 

Potency/ self  efficacy 

Team  spirit 

Contingency 

factors 

Variables  impinging  from 
sources  internal  and  external  to 
the  team 

Team  application/  mission 
Resource  availability 

Procedural  requirements 

Rules  on  operation,  managing, 
or  decision-making. 
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Appendix  B:  Definitions 

(Vaughan  &  Hogg,  1995) 


Evaluation  apprehension- 

The  physical  presence  of  members  of  the  same  species  causes  drive  because  people 
have  learned  to  be  apprehensive  about  being  evaluated. 

Socialising- 

Dynamic  relationship  between  the  group  and  its  members  that  describes  the  passage  of 
members  through  a  group  in  terms  of  commitment  and  changing  roles. 

Conformity  pressures- 

The  pressure  to  conform  to  group  norms  in  order  to  identify  with  and  be  accepted  by 
that  group. 

Social  loafing- 

A  reduction  in  individual  effort  when  working  on  a  collective  task  (in  which  one's 
outputs  are  pooled  with  those  of  other  group  members)  compared  to  when  working 
either  alone  or  coactively. 

Domination  due  to  status  imbalance- 

Different  group  members  have  a  different  perceived  status  in  the  group.  Status  usually 
depends  on  their  consensual  prestige  and  their  tendency  to  initiate  ideas  and  activities 
that  are  adopted  by  the  group.  High  status  members  tend  to  dominate  idea  formation 
in  groups. 

Free  riding- 

When  a  group  member  is  avoiding  costly  obligations  and  group  membership  and 
allowing  other  members  to  incur  the  costs. 

Group  think- 

A  mode  of  thinking  in  highly  cohesive  groups  in  which  the  desire  to  reach  unanimous 
agreement  overrides  the  motivation  to  adopt  proper  rational  decision-making 
procedures. 
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